-- Logs begin at Wed 2022-08-24 02:45:42 UTC, end at Wed 2022-08-24 04:06:07 UTC. -- Aug 24 03:21:20.663191 dv-cinder-854262-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 24 03:21:20.670178 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 24 03:21:20.672186 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Aug 24 03:21:20 2022] *** Aug 24 03:21:20.672186 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 24 03:21:20.672186 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: nodename: dv-cinder-854262-1-iscsi Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: machine: x86_64 Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: clock source: unix Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: pcre jit disabled Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: detected number of CPU cores: 4 Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: current working directory: / Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: detected binary path: /usr/bin/uwsgi-core Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: your processes number limit is 47680 Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: your memory page size is 4096 bytes Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: detected max file descriptor number: 2048 Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: lock engine: pthread robust mutexes Aug 24 03:21:20.672416 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: thunder lock: enabled Aug 24 03:21:20.673123 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 24 03:21:20.673123 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 24 03:21:20.712308 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: Python main interpreter initialized at 0x55bbd769e9c0 Aug 24 03:21:20.712308 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: python threads support enabled Aug 24 03:21:20.712308 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: your server socket listen backlog is limited to 100 connections Aug 24 03:21:20.712308 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: your mercy for graceful operations on workers is 90 seconds Aug 24 03:21:20.712719 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: mapped 403077 bytes (393 KB) for 2 cores Aug 24 03:21:20.712804 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: *** Operational MODE: preforking *** Aug 24 03:21:20.713049 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: *** uWSGI is running in multiple interpreter mode *** Aug 24 03:21:20.713049 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: spawned uWSGI master process (pid: 81512) Aug 24 03:21:20.713255 dv-cinder-854262-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 24 03:21:20.714388 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: spawned uWSGI worker 1 (pid: 81521, cores: 1) Aug 24 03:21:20.714388 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: spawned uWSGI worker 2 (pid: 81522, cores: 1) Aug 24 03:21:20.714388 dv-cinder-854262-1-iscsi devstack@keystone.service[81512]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 24 03:21:24.992769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:21:24.992769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: from cryptography.utils import int_from_bytes Aug 24 03:21:24.993564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:21:24.993564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: from cryptography.utils import int_from_bytes Aug 24 03:21:24.993989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:21:24.993989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: from cryptography.utils import int_from_bytes Aug 24 03:21:24.994103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 24 03:21:24.994103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: from cryptography.utils import int_from_bytes Aug 24 03:21:25.043844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 03:21:25.044028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 03:21:25.044212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] command line args: [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 03:21:25.044361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 24 03:21:25.044488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 03:21:25.044576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 24 03:21:25.044650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ================================================================================ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 03:21:25.044733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] command line args: [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 24 03:21:25.044910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] admin_token = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.044997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 24 03:21:25.045160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_dir = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ================================================================================ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 24 03:21:25.045331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] admin_token = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] config_source = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_dir = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] debug = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.045934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] config_source = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] debug = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.046963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_config_append = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_dir = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_config_append = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.047983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_dir = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.048964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.049951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.050921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.91/identity {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.051879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] publish_errors = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.91/identity {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] publish_errors = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.052960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] standard_threads = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] standard_threads = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.053915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_journal = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_json = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_stderr = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_journal = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] use_syslog = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_json = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.054918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_stderr = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.055028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.055107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] use_syslog = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.055200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.055272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 24 03:21:25.055463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.055557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.055702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.055797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.055950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.056913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.057991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.058957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.059975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.060920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.061842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.062506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.062626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.062743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.062857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.062971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.063914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.064994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.065935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066410 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.066996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067806 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.067927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.068967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.069930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.070886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.071959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.072944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.073994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.074944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.075984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.076904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.077020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.077135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.077249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.077365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.077479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.078935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.079991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.external = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.080924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.external = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.password = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.081909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] auth.token = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.password = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.082950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] auth.token = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.083891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.084938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.085891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.086962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.087913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.088879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.089845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.090922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.091979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.092988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.093988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.094834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.095968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.096914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.097984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.098947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.099941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.100905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.101965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.102994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.103875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.104916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.105974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.106917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.107039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.107155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.107278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.107395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.107510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.107625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.108270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.108403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.108522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.108637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.108753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.109362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.109478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.109601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.109718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.109833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.109949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.110942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.111888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.112979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.113923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.114948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.115889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.116885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.117935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.118945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.119999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.120812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.121984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.122851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.123953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.124068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.124184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.124300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.125943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.126810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.127971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.128932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.129883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.130963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81522) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 03:21:25.131657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 24 03:21:25.131657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81521) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 24 03:21:25.159934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.160155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.160244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.160385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.160472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.160615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.168844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.169071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.198592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.198820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.198909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.199135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.253035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.253392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.302229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.302464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.302883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.303129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.460494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.460687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.460837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.460926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.461636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.485573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.485691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.485801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.485940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.486919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.487050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.497921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.498286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.510749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.510869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.510950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.511965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.512050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81522) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.512180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81521) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 24 03:21:25.528603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.528816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:21:25.529161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.529248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.529363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:21:25.529467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:21:25.529642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.529727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.530163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 24 03:21:25.530270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.530467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.531231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.531323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.531519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.531767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.532037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.540868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.541150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.541420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.541510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.541704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.541805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.541984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.542301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.549275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.549851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.552205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.552494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.552778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.552861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.553065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.553984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.554165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.554277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.554470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.554574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.554752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.554857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.555131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.555212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.555414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.555691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.577745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.578446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.579265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.579555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.579919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.580209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.584232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.584522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.584872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.584971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.585188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.585477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.591198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.591489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.591894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.592188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.596126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.596421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.596709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.596849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.596997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.597992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.598101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.598298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.598578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.598854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.615916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.616411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.616668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.616963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.620105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.620897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.623343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.623646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.624148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.624968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.628397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.629218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.631565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.632385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.634828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.635138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.635428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.635683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.635771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.635990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.636083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.636281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.636365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.636576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.636863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.637157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.652703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.652992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.653262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.653530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.653698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.653993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.654289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.654560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.660713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.661776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.662263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.662548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.663331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.663611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.665170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.665459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.665752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.666056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.666287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.666374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.666579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.666878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.667180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.667464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.678203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.678503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.679232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.679856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.680736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.681526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.695790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.696573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.698854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.699659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.701762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.702662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.703848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.704620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.705334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.705620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.705919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.706101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.706415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.706721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.712428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.713218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.713884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.714704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.716838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.717135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.717424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.717686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.717773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.717984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.718328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.718623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.725702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.725995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.726306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.726635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.726931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.727218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.733213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81522) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.733503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.733811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.734098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.734201 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81521) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 24 03:21:25.734406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.734490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.734694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.734778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.734971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.735891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81522) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.736981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.737162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.737448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.737713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81521) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 24 03:21:25.771932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bbd769e9c0 pid: 81522 (default app) Aug 24 03:21:25.773105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55bbd769e9c0 pid: 81521 (default app) Aug 24 03:21:25.775393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c24e22-03ee-446a-a836-c3e4ae1e96b1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:25.775684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c24e22-03ee-446a-a836-c3e4ae1e96b1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:25.775958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c24e22-03ee-446a-a836-c3e4ae1e96b1 None None] PATH_INFO: `/v3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:25.780365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1/1] 10.222.0.91 () {50 vars in 791 bytes} [Wed Aug 24 03:21:25 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:32.304528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:32.305042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:32.305042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:32.309335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1/2] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:32.316670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:32.317049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:32.317213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:32.389272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: INFO dbcounter [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] Registered counter for database keystone Aug 24 03:21:32.408938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG oslo_db.sqlalchemy.engines [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81522) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 03:21:32.415173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writer thread running {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 03:21:32.596161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG passlib.handlers.bcrypt [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81522) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 03:21:32.596581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG passlib.handlers.bcrypt [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81522) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 03:21:32.653560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:32.655497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:32.783177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 2/3] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:32 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 03:21:33.237055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:33.305985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: INFO dbcounter [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] Registered counter for database keystone Aug 24 03:21:33.328138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG oslo_db.sqlalchemy.engines [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81521) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 24 03:21:33.334443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writer thread running {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 24 03:21:33.454421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa8b853-5aa4-466e-a3d0-6f6fa2d9c26f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:33.458712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:33.459344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:33.459932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:33.459932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:33.468500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:33.519526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 03:21:33.519872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7e5b1b-eb7f-4fdc-96c3-e0766fabad06 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:33.529338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 2/4] 10.222.0.91 () {58 vars in 1324 bytes} [Wed Aug 24 03:21:33 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:21:33.536219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:33.550388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea068fd-1a8d-4a29-ad51-c1ecae62ac17 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:33.554813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:33.555908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:33.556187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:33.556455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] PATH_INFO: `/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:33.558023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] RBAC: Authorizing `identity:get_project(project_id=24a8303d4cc94c8ebcfdb44c0be112a5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:33.611768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING keystone.common.rbac_enforcer.enforcer [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 24 03:21:33.612219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d892f2d-8a86-4195-ab87-59289d2e1e76 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:33.617089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 3/5] 10.222.0.91 () {56 vars in 1207 bytes} [Wed Aug 24 03:21:33 2022] GET /identity/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:35.476337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa3036-f643-4403-b4f5-19357bbd32a7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:35.476859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa3036-f643-4403-b4f5-19357bbd32a7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:35.476945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa3036-f643-4403-b4f5-19357bbd32a7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:35.481023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 3/6] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:35.486529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:35.486958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:35.487080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:35.507225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:35.508762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:35.517150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 4/7] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 24 03:21:35.788524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa3036-f643-4403-b4f5-19357bbd32a7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:35.801779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa3036-f643-4403-b4f5-19357bbd32a7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:35.806124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37750b02-a77b-4811-9371-e2e507090d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:35.806772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750b02-a77b-4811-9371-e2e507090d76 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:35.807054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750b02-a77b-4811-9371-e2e507090d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:35.807326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37750b02-a77b-4811-9371-e2e507090d76 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:35.816408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750b02-a77b-4811-9371-e2e507090d76 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:35.818211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37750b02-a77b-4811-9371-e2e507090d76 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:35.827245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 4/8] 10.222.0.91 () {58 vars in 1324 bytes} [Wed Aug 24 03:21:35 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:21:35.833027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:35.842460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bcff8be-1e40-4620-b68f-252929c44e04 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:35.846868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63368b7-6da9-4156-8453-4951b72c98b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:35.847500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63368b7-6da9-4156-8453-4951b72c98b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:35.847780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63368b7-6da9-4156-8453-4951b72c98b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:35.848053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63368b7-6da9-4156-8453-4951b72c98b0 admin admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:35.849382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63368b7-6da9-4156-8453-4951b72c98b0 admin admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:35.851310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63368b7-6da9-4156-8453-4951b72c98b0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:35.852858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 5/9] 10.222.0.91 () {56 vars in 1198 bytes} [Wed Aug 24 03:21:35 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:37.797772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a94cec-e296-4c6a-b28a-71f12068f1fd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:37.798369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a94cec-e296-4c6a-b28a-71f12068f1fd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:37.798369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a94cec-e296-4c6a-b28a-71f12068f1fd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:37.799866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 5/10] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:37.805199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:37.805626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:37.805883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:37.825370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:37.826574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:37.860982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde2e7-6e3a-4228-aaab-873705d5dc78 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:37.861276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde2e7-6e3a-4228-aaab-873705d5dc78 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:37.861548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cde2e7-6e3a-4228-aaab-873705d5dc78 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:37.862915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 6/11] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:37.869521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:37.869947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:37.870130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:37.912713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG passlib.handlers.bcrypt [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81521) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 24 03:21:37.913125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG passlib.handlers.bcrypt [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81521) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 24 03:21:37.913455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 6/12] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:37.959327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:37.961070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:38.059012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 7/13] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:21:38.179723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.203952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.285153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb2acb7-9619-449b-9c96-1ba4f6bb2876 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.319934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.320498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.320774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.321046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.324914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.334380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:38.343654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:38.344795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:21:38.345668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 7/14] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:38.350945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.430640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c41d2-42e0-491c-b42d-be5a38695bde None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.433167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.435701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.457471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c902f0f-43af-4eed-97d9-d1e980bbf932 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.472101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.473173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.473608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.474008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.493121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.493742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.494023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.494343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.495070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:38.495338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:38.496742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:38.509300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 8/15] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:38.514941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.563427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:38.565244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:38.622341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a6a8f-5caf-4da3-9411-bb81395d871a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.663814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.664818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.665284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.665742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.667302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:38.668850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:38.671676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 9/16] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:38.675722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 8/17] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:21:38.679759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.681231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.684257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.696919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.790389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475749f1-c40f-4c6b-96c0-b348303bae5f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.791616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc5574-f363-4edb-ad77-1e666518ae68 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.833339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.834020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.834593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.834730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.834850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.834966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.835090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.835229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.836313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:38.837824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:38.839457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 10/18] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:38.845849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.855836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:38.857346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:21:38.876171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:38.877151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:21:38.878001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:21:38.878001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:38.878001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:21:38.878001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 9/19] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:21:38.883100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:38.931094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.944014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:38.953796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f53aeb4-12dd-49e6-b8f3-2fba06fdc86a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.992941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:38.993302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0794397b-12d2-4a63-bee3-b34cc0237f1a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:38.993571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:38.993954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:38.994395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:38.995636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.domain.id=default, user.id=7d825b9d979b45a095dd6e2cabfe9971) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:38.996758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:38.997403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd42d0c5-6de4-478f-9230-17e2b009afcb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:39.005190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 11/20] 10.222.0.91 () {56 vars in 1322 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/role_assignments?user.id=7d825b9d979b45a095dd6e2cabfe9971&scope.domain.id=default&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 233 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:39.035620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:39.036687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:39.037170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:39.037640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:39.038642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:39.038915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:39.040380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d887ab6-9b85-4aaa-8e6d-4b7f5b6b6949 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:39.062807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 10/21] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:21:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:41.132444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1d13c-fa61-4375-bbf2-a447c1b1309b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:41.132962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1d13c-fa61-4375-bbf2-a447c1b1309b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:41.133069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1d13c-fa61-4375-bbf2-a447c1b1309b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:41.134755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 12/22] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:41.140082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065640-b18d-4802-8988-67bfec8d04e3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:41.140564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065640-b18d-4802-8988-67bfec8d04e3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:41.140861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1065640-b18d-4802-8988-67bfec8d04e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:41.171857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e1065640-b18d-4802-8988-67bfec8d04e3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:41.173135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e1065640-b18d-4802-8988-67bfec8d04e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:41.269379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 11/23] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:41.580246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d93ee-d0f9-45a4-883f-08df4040d887 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:41.580639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d93ee-d0f9-45a4-883f-08df4040d887 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:41.580829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d93ee-d0f9-45a4-883f-08df4040d887 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:41.582686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 13/24] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:41.588714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f04ce-447f-4d9b-b005-854f8861b7db None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:41.589000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f04ce-447f-4d9b-b005-854f8861b7db None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:41.589278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114f04ce-447f-4d9b-b005-854f8861b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:41.616579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-114f04ce-447f-4d9b-b005-854f8861b7db None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:41.617800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-114f04ce-447f-4d9b-b005-854f8861b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:41.684608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:41.718821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 12/25] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:41.834178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d93ee-d0f9-45a4-883f-08df4040d887 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:41.959773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede2bea-529b-4fbb-bcf8-a23fabca502d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:41.960466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede2bea-529b-4fbb-bcf8-a23fabca502d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:41.961034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dede2bea-529b-4fbb-bcf8-a23fabca502d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:41.962693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 13/26] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:41.969463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11b78e-d427-454b-9e21-49f1efe4f61d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:41.970089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11b78e-d427-454b-9e21-49f1efe4f61d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:41.970600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11b78e-d427-454b-9e21-49f1efe4f61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:41.978212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d93ee-d0f9-45a4-883f-08df4040d887 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:42.007985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4b11b78e-d427-454b-9e21-49f1efe4f61d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:42.009205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4b11b78e-d427-454b-9e21-49f1efe4f61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:42.061544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:42.062309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.062596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.062874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.085542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:42.091116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:42.092131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:42.093449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:21:42.095671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 14/27] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:21:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:42.101676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:42.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:42.146607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 14/28] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:42.239771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3bb3-a8a3-4440-9ec6-d0d97bea7bed None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.240415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3bb3-a8a3-4440-9ec6-d0d97bea7bed None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.240920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bb3bb3-a8a3-4440-9ec6-d0d97bea7bed None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.242627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 15/29] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:42.247340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bd04a-4dce-4539-924c-92231fef14de None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:42.251191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.251745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.252220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.291963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:42.292599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.292887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.293165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.293858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:42.294165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:42.295644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:42.303537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:42.305175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:42.311141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 15/30] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:42.313036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:42.390718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:42.441409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27680aa2-1b10-4449-9fc7-10137c30bfad None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:42.490998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 16/31] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:21:42.494291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:42.496374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:42.499037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.500045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.500506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.520465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:42.522355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:21:42.539851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:42.540866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:21:42.541705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:21:42.541705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:21:42.541705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:21:42.544196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 16/32] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:42.545489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:42.549701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:42.702164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8711c48-66fb-4594-8386-7370e1c32360 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:42.721208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfcc410-9661-4a83-8580-be305a40036c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:42.741419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:42.751707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:42.752728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.753253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.753737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.755625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:42.757360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:42.759540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 17/33] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:42 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:42.765034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:42.765222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f58add-660f-47a7-9b2e-b9d5c7d9811a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:42.765222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f58add-660f-47a7-9b2e-b9d5c7d9811a None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.765222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f58add-660f-47a7-9b2e-b9d5c7d9811a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.765222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f58add-660f-47a7-9b2e-b9d5c7d9811a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.765521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f58add-660f-47a7-9b2e-b9d5c7d9811a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:42.765521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f58add-660f-47a7-9b2e-b9d5c7d9811a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:42.881283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2416c-c5cc-4e22-9eb1-9bfd3846b8c2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:42.885204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:42.906040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 17/34] 10.222.0.91 () {60 vars in 1131 bytes} [Wed Aug 24 03:21:42 2022] POST /identity/v3/roles => generated 219 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:42.906040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a22ef7-818b-4626-b5cc-b773112a0bf4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.906040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a22ef7-818b-4626-b5cc-b773112a0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.906040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a22ef7-818b-4626-b5cc-b773112a0bf4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.906040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 18/35] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:42.906040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a22ef7-818b-4626-b5cc-b773112a0bf4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:42.934553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:42.935560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:42.936033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:42.936491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:42.937510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:42.938103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:42.939805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:42.955124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 18/36] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:21:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:42.956484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:43.081420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a22ef7-818b-4626-b5cc-b773112a0bf4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:43.094969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a619e2-4d12-4fe8-821a-fd5b05bcdf8d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:43.136376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99b58f4-e9fe-4a53-8e76-dbc986c3c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:43.137019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b58f4-e9fe-4a53-8e76-dbc986c3c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.137309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b58f4-e9fe-4a53-8e76-dbc986c3c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.137587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99b58f4-e9fe-4a53-8e76-dbc986c3c7c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.140871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b58f4-e9fe-4a53-8e76-dbc986c3c7c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:43.142682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99b58f4-e9fe-4a53-8e76-dbc986c3c7c2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:43.144502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 19/37] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:43.146313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.146313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.146585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.151605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:43.152797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.153207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.153631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.154560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:43.157376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:43.175906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:43.177732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:43.225436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:43.258220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 20/38] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:43.258220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:43.330776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 19/39] 10.222.0.91 () {60 vars in 1131 bytes} [Wed Aug 24 03:21:42 2022] POST /identity/v3/roles => generated 225 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:43.332835 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:43.380026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:43.388832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb5bf8f-df12-4fcb-b9c9-d6789d202baa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:43.440756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8e3f23-92ea-475e-89ff-44e5897b41a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:43.441809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e3f23-92ea-475e-89ff-44e5897b41a1 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.442152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e3f23-92ea-475e-89ff-44e5897b41a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.442507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8e3f23-92ea-475e-89ff-44e5897b41a1 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.443966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e3f23-92ea-475e-89ff-44e5897b41a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:43.445923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8e3f23-92ea-475e-89ff-44e5897b41a1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:43.494097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 21/40] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:21:43 2022] POST /identity/v3/projects => generated 301 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:43.495672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4446d94-3f6c-47ed-8938-53b9cf263115 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:43.495787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb198d1-395a-44b4-ba84-8e7b9bbe5560 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.495972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb198d1-395a-44b4-ba84-8e7b9bbe5560 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.496260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb198d1-395a-44b4-ba84-8e7b9bbe5560 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.497672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 22/41] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:43.506478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.506803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.507040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.528364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:43.530491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:43.548559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:43.549794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.550357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.551041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] PATH_INFO: `/v3/domains/default/users/7d825b9d979b45a095dd6e2cabfe9971/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.556714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7d825b9d979b45a095dd6e2cabfe9971, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:43.559158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:43.567840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:43.586463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 20/42] 10.222.0.91 () {58 vars in 1361 bytes} [Wed Aug 24 03:21:43 2022] PUT /identity/v3/domains/default/users/7d825b9d979b45a095dd6e2cabfe9971/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:21:43.635818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:43.642401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 23/43] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:43.755963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:43.777719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe0d954-d900-426e-89e4-cca10e3f227a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:43.822861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:43.823835 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:43.824127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:43.824371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:43.825108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:43.827008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:43.850957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 21/44] 10.222.0.91 () {60 vars in 1131 bytes} [Wed Aug 24 03:21:43 2022] POST /identity/v3/roles => generated 223 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:21:43.953612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:44.166838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6342cf0-332f-488e-ba9e-e83ca7004ebc None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:44.201575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118849af-0aa3-41a2-956d-518a0f82f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:44.202268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118849af-0aa3-41a2-956d-518a0f82f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:44.202542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118849af-0aa3-41a2-956d-518a0f82f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:44.202815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118849af-0aa3-41a2-956d-518a0f82f9c5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:44.203924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118849af-0aa3-41a2-956d-518a0f82f9c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:44.205365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118849af-0aa3-41a2-956d-518a0f82f9c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:44.206960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 24/45] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:44.212560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:44.312365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af46fa7-13a7-4b00-980b-f472d7d63011 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:44.345020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7f0d6e-0ee7-459c-9cca-93b335c069e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:44.345633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f0d6e-0ee7-459c-9cca-93b335c069e9 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:44.345915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f0d6e-0ee7-459c-9cca-93b335c069e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:44.346212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7f0d6e-0ee7-459c-9cca-93b335c069e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:44.346857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f0d6e-0ee7-459c-9cca-93b335c069e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:44.348296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7f0d6e-0ee7-459c-9cca-93b335c069e9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:44.365690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:44.377380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 22/46] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:21:44 2022] POST /identity/v3/projects => generated 312 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:45.568265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11994501-0a84-49bc-a99e-bce440876a5b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:45.568806 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11994501-0a84-49bc-a99e-bce440876a5b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:45.569056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11994501-0a84-49bc-a99e-bce440876a5b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:45.570430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 25/47] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:45.575977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:45.576382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:45.576539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:45.595306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:45.596819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:45.666994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 23/48] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:45.936524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11994501-0a84-49bc-a99e-bce440876a5b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:45.964533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:46.041099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11994501-0a84-49bc-a99e-bce440876a5b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:46.075415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:46.076045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.076326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.076601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.087059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:46.091745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:46.092623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:21:46.094253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 26/49] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:21:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:46.100810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:46.201147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48563079-2878-47ef-8ee5-6a695b4de2b1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:46.228266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:46.237256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:46.237858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.238141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.238450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.239120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:46.239425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:46.240940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30348ac7-42ed-4a67-90d2-575d39ab1d2c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:46.252948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 24/50] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:46.258600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:46.292697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85111a53-e2b6-4981-85a3-5e615244dd1c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.292984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85111a53-e2b6-4981-85a3-5e615244dd1c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.293261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85111a53-e2b6-4981-85a3-5e615244dd1c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.294624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 25/51] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:46.299710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.300005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.300285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.318032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:46.319284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:46.345184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:46.360572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf82379b-c9fe-4656-91ec-91c69dde345c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:46.388593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 26/52] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:21:46.394885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:46.395517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.395800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.396095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.397208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:46.398737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:46.400311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 27/53] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:46 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:46.406792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:46.503628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd9c917-fdf6-4d91-90a5-62e72f309e4e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:46.522087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:46.538733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:46.539369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.539649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.539923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.541044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:46.542650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:46.544377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 27/54] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:46.550116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:46.650286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4121bcd-54ae-4aaa-89dc-37a13d189560 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:46.660536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:46.684813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:46.685423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.685708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.685989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.687288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.domain.id=default, user.id=7d825b9d979b45a095dd6e2cabfe9971) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:46.687596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:46.689067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:46.695023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 28/55] 10.222.0.91 () {56 vars in 1322 bytes} [Wed Aug 24 03:21:46 2022] GET /identity/v3/role_assignments?user.id=7d825b9d979b45a095dd6e2cabfe9971&scope.domain.id=default&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 530 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:46.872032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b0f924-6e93-4882-bd09-6adc668c4193 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:46.906492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9132d6-0ce3-4f5b-8bdd-bc950f83bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:46.907146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9132d6-0ce3-4f5b-8bdd-bc950f83bce7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:46.907435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9132d6-0ce3-4f5b-8bdd-bc950f83bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:46.907716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9132d6-0ce3-4f5b-8bdd-bc950f83bce7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:46.908852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9132d6-0ce3-4f5b-8bdd-bc950f83bce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:46.910342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9132d6-0ce3-4f5b-8bdd-bc950f83bce7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:46.912083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 28/56] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:46.917646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:46.938066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:47.020147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1031d6c-04b6-4bbc-aabb-c6ea454b1cc4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:47.053340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc069739-6f32-41ff-b0c6-09f1052c859c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:47.053945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069739-6f32-41ff-b0c6-09f1052c859c None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:47.054253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069739-6f32-41ff-b0c6-09f1052c859c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:47.054551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc069739-6f32-41ff-b0c6-09f1052c859c None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:47.055198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc069739-6f32-41ff-b0c6-09f1052c859c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:47.056656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc069739-6f32-41ff-b0c6-09f1052c859c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:47.082146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 29/57] 10.222.0.91 () {60 vars in 1140 bytes} [Wed Aug 24 03:21:46 2022] POST /identity/v3/projects => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:48.997757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92158852-3cf9-4cd9-a998-2cd0722b2843 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:48.998312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92158852-3cf9-4cd9-a998-2cd0722b2843 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:48.998312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92158852-3cf9-4cd9-a998-2cd0722b2843 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:49.000175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 29/58] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:49.006256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:49.006727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:49.006832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:49.025695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:49.026964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:49.092244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:49.097578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 30/59] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:21:49.365839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92158852-3cf9-4cd9-a998-2cd0722b2843 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:49.427359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:49.475829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92158852-3cf9-4cd9-a998-2cd0722b2843 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:49.512194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4049e00-f339-4628-a0c7-e7a0e6b394aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:49.512827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4049e00-f339-4628-a0c7-e7a0e6b394aa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:49.513106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4049e00-f339-4628-a0c7-e7a0e6b394aa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:49.513376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4049e00-f339-4628-a0c7-e7a0e6b394aa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:49.515136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4049e00-f339-4628-a0c7-e7a0e6b394aa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:49.516613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4049e00-f339-4628-a0c7-e7a0e6b394aa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:49.518273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 30/60] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:49.524093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:49.627538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0d5e-5256-4b16-809a-ec4514b75aad None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:49.662425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6be3e9-48b8-4e71-a3cf-1c93eb27e214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:49.663050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6be3e9-48b8-4e71-a3cf-1c93eb27e214 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:49.663329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6be3e9-48b8-4e71-a3cf-1c93eb27e214 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:49.663604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6be3e9-48b8-4e71-a3cf-1c93eb27e214 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:49.664281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6be3e9-48b8-4e71-a3cf-1c93eb27e214 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:49.665760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6be3e9-48b8-4e71-a3cf-1c93eb27e214 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:49.693446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 31/61] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:21:49 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:52.868664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf644-9fb5-4dee-b4c0-f4925ff76cfc None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:52.869532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf644-9fb5-4dee-b4c0-f4925ff76cfc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:52.869972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dbf644-9fb5-4dee-b4c0-f4925ff76cfc None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:52.871516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 31/62] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:52.877404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861d103-ead9-4718-b566-2f824d9fc045 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:52.878279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861d103-ead9-4718-b566-2f824d9fc045 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:52.878723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e861d103-ead9-4718-b566-2f824d9fc045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:52.907404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e861d103-ead9-4718-b566-2f824d9fc045 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:52.908879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e861d103-ead9-4718-b566-2f824d9fc045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:52.975473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:53.004863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 32/63] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:21:53.130718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f7f86-dd78-4b11-b4c8-d4e69641aa13 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.131493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f7f86-dd78-4b11-b4c8-d4e69641aa13 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.131966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612f7f86-dd78-4b11-b4c8-d4e69641aa13 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.133819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 32/64] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:53.141662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2e7c-dbb7-4586-b591-5542e95dbcd8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.142604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2e7c-dbb7-4586-b591-5542e95dbcd8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.143351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2e7c-dbb7-4586-b591-5542e95dbcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.180312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-939e2e7c-dbb7-4586-b591-5542e95dbcd8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:53.181541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-939e2e7c-dbb7-4586-b591-5542e95dbcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:53.260612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcb141-15e0-4235-bbce-268a87196026 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.261379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcb141-15e0-4235-bbce-268a87196026 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.262001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bcb141-15e0-4235-bbce-268a87196026 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.267670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 33/65] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:53.273099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.273741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.274258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.279738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 33/66] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:53.305546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93032c-e30a-4da1-b976-cbc51e9aee0f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.306219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93032c-e30a-4da1-b976-cbc51e9aee0f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.306709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe93032c-e30a-4da1-b976-cbc51e9aee0f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.307998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:53.308394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 34/67] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:53.309665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:53.314772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.315155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.315525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.347295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:53.348692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:53.420670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 34/68] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:53.485490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 35/69] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:53.576430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:53.603944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:53.665665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:53.680616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:53.726124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad90d17-288b-494d-91f6-7dfeb1b114cc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:53.764294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:53.765481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.765763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.766038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.778021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:53.786225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a522fe-05e5-4204-a357-c7d75304ce3a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:53.786415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:53.787409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 35/70] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:21:53.788288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:53.825967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:53.826629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.826903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.827699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.839192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:53.844499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:53.845486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:21:53.846713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:21:53.850410 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 36/71] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:21:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:21:53.851300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:53.910401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc16997e-6b2a-4e6b-bec1-599b81c4c10b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:53.923527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:53.945827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:53.946402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:53.946727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:53.947300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:53.961144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:53.962879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad95eade-252f-4e04-b034-8dc50e661a52 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:53.965937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:53.966840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:21:53.967607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 36/72] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:21:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:53.968912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:53.970707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.000228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.000853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.001137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.001413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.002090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:54.002433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.003855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.012516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 37/73] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:21:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.013020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.076041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80dbf8-69c1-41e5-ad63-e65a73587233 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.114681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2134bf2a-6632-40b8-a1aa-589f7376567e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.115331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134bf2a-6632-40b8-a1aa-589f7376567e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.115623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134bf2a-6632-40b8-a1aa-589f7376567e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.115901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2134bf2a-6632-40b8-a1aa-589f7376567e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.126064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b50c9-24c4-4956-ab9d-33ef30adcc78 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.126718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2134bf2a-6632-40b8-a1aa-589f7376567e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2134bf2a-6632-40b8-a1aa-589f7376567e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:54.131922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:54.132803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:21:54.133989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 37/74] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:54.134926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242e6b2-444f-4233-a9b2-d35b64bad183 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.134926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242e6b2-444f-4233-a9b2-d35b64bad183 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.135233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8242e6b2-444f-4233-a9b2-d35b64bad183 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.136861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 38/75] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:54.139256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242e6b2-444f-4233-a9b2-d35b64bad183 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.166319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.167131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.167503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.167869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.168821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:54.169332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.171696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.175995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.181079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 38/76] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.181916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.217112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.258105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8242e6b2-444f-4233-a9b2-d35b64bad183 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.294019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.294666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.294950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.295114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69f7b0-d7cb-4059-9459-3a0eca6ef889 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.295227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.295877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:54.297006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.298527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.307900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 39/77] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.308730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.333239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0a2912-64a7-4927-8e1f-27a11f4c967d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.333869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a2912-64a7-4927-8e1f-27a11f4c967d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.334149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a2912-64a7-4927-8e1f-27a11f4c967d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.334477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0a2912-64a7-4927-8e1f-27a11f4c967d None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.349114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a2912-64a7-4927-8e1f-27a11f4c967d None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0a2912-64a7-4927-8e1f-27a11f4c967d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.352351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 39/78] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.353585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.353869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.354148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.371602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:54.372831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:54.410039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.418723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e991b95-ab49-470f-b2d5-f4e798e5b007 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.446342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 40/79] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:54.447251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.467737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.469286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.469913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.470211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.470497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.471439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:54.471543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.473086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.482582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 40/80] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.483018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.565211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc48a4a-c84d-4e40-8d53-8d477bd488e0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.596638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9b4f6b-6bb0-4a3f-97b1-21f25447e0ec None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.604696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.605369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.606007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.606332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.607802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.609493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.615695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 41/81] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.615695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.637251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.638059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.638385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.638659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.640397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.641890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.643727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 41/82] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.644284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.672471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.712525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.728799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d047ef0-1972-4182-914c-b1c2380d06d9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.754561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d8ef11-70a6-42c8-aae9-2239bbd05f66 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.763590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.764202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.764487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.765327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.789301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.789947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.790257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.790533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.791722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.793197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.794874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 42/83] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:54.795653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:54.837634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:54.839174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:54.897770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.907183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038c1c2-ecd4-48f1-aae6-57f0d1a4d965 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:54.928359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:54.942883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:54.943523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:54.943806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:54.944083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:54.946623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 42/84] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:21:54.947653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.021021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.023041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.081812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aef037-7c7b-494f-84ad-4ecb8fde96b2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.117354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.128307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 43/85] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:21:55.128774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.128969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.129396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.129679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.129955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.190359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.204834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.206398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.232606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debc1a1c-d157-4894-b50c-125610e87ed2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.276798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.277794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.278283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.278771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.280636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.282414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.284995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 44/86] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:55.285875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.317186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 43/87] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:21:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:21:55.318466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.364383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.415230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8afa6-4103-4145-a508-8046b2582f78 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.431541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.451444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2dbeed-fe68-4215-bb3f-98456dc228b3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.463084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.463760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.464126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.464451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.496736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.497492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.497775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.498047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.499244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.500714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.502833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 44/88] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:55.503240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.555360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.556792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.621312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.624813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43535be5-d934-4064-930a-1c682d84c524 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.663209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.663852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.664133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.664410 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.665589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.667091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.668935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 45/89] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:55.674217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 45/90] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:21:55.674374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.674451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.700556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.783012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fa624-2fa5-4900-abcf-71d60d661673 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.786302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe6e16e-c0af-486f-90b5-fb961b09c982 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.820938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.821473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.821757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.822013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:55.822126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.822712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.822810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:55.823004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:55.823305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:55.824175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.824441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:55.825861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:55.833097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 46/91] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:55.833097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.847426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 46/92] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:21:55 2022] POST /identity/v3/users => generated 289 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:55.848283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:55.860997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.955384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d08a2e-3dca-4715-b6e0-d1021b28adcd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:55.972940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:55.999990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e372c-e2d0-40d9-8f3a-ec294a87b938 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:56.002887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:56.003512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:56.004273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:56.004273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:56.004839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=785cf2fb0f794ab8b0d643d7797ff5dc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:56.005143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:56.006995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:56.016187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 47/93] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/role_assignments?user.id=785cf2fb0f794ab8b0d643d7797ff5dc&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=d89a893a140749d39b9cecdedae58d78 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:56.017100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:56.044104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:56.044770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:56.045051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:56.045338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] PATH_INFO: `/v3/projects/b930502887274f279815f7b672deeabd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:56.047708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] RBAC: Authorizing `identity:get_project(project_id=b930502887274f279815f7b672deeabd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:56.049220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:56.051244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 47/94] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:21:55 2022] GET /identity/v3/projects/b930502887274f279815f7b672deeabd => generated 312 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:56.054236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:56.167544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:56.182380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ca027-aa0b-4706-8608-f83ef99d65a5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:56.215012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b71f837-96e0-4cff-9caa-aea8f3ef1a40 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:56.234434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:56.235070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:56.235356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:56.235638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:56.237222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] RBAC: Adding query filter params (role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=785cf2fb0f794ab8b0d643d7797ff5dc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:56.237222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:56.239051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:56.246693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 48/95] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:21:56 2022] GET /identity/v3/role_assignments?user.id=785cf2fb0f794ab8b0d643d7797ff5dc&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:56.247566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:56.262822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:56.263452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:56.263736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:56.264010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:56.265384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=7d825b9d979b45a095dd6e2cabfe9971) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:56.265700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:56.267277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d05a5f-a299-4394-abab-5bf5f2f783b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:56.275469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 48/96] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:21:56 2022] GET /identity/v3/role_assignments?user.id=7d825b9d979b45a095dd6e2cabfe9971&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:56.381030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a662e-d08a-41bc-9f59-54b710848b4d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:56.425059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:56.425713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:56.426021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:56.426310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:56.427100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=b930502887274f279815f7b672deeabd, user.id=785cf2fb0f794ab8b0d643d7797ff5dc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:56.427415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:56.429022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c79827-0ac9-409e-aedb-24ce58240a8f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:56.436140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 49/97] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:21:56 2022] GET /identity/v3/role_assignments?user.id=785cf2fb0f794ab8b0d643d7797ff5dc&scope.project.id=b930502887274f279815f7b672deeabd&role.id=d89a893a140749d39b9cecdedae58d78 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:58.692962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b6c88-12fe-4d3f-9d74-ebb84c9ec735 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:58.694062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b6c88-12fe-4d3f-9d74-ebb84c9ec735 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:58.694587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b6c88-12fe-4d3f-9d74-ebb84c9ec735 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:58.696175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 49/98] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:58.705906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696249c-cc28-4377-91e0-d35b9017420d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:58.707322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696249c-cc28-4377-91e0-d35b9017420d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:58.708060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8696249c-cc28-4377-91e0-d35b9017420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:58.738866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8696249c-cc28-4377-91e0-d35b9017420d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:58.741413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8696249c-cc28-4377-91e0-d35b9017420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:58.763243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee179-1d24-4d62-af99-5ef47fff6035 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:58.764213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee179-1d24-4d62-af99-5ef47fff6035 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:58.765585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ee179-1d24-4d62-af99-5ef47fff6035 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:58.768405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 50/99] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:21:58.774738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a4301-d771-4fc6-9db5-82a703b32819 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:58.775566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a4301-d771-4fc6-9db5-82a703b32819 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:58.776324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a4301-d771-4fc6-9db5-82a703b32819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:58.788235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:58.829113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c08a4301-d771-4fc6-9db5-82a703b32819 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:58.835057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c08a4301-d771-4fc6-9db5-82a703b32819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:58.846473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:58.940774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 50/100] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:58.982088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 51/101] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:21:59.049920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890178d-c7e9-44d4-b02f-0a006c9326e7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.050587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890178d-c7e9-44d4-b02f-0a006c9326e7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.051066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c890178d-c7e9-44d4-b02f-0a006c9326e7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.052647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 51/102] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:59.058070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38504e-959b-4ef7-9705-d9428e06a468 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.058634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38504e-959b-4ef7-9705-d9428e06a468 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.059114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c38504e-959b-4ef7-9705-d9428e06a468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.086469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7c38504e-959b-4ef7-9705-d9428e06a468 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:59.088051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c38504e-959b-4ef7-9705-d9428e06a468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:59.192895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 52/103] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:59.199759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bf372-b531-48e0-a50f-3576938e1d8c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.200053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bf372-b531-48e0-a50f-3576938e1d8c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.200329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087bf372-b531-48e0-a50f-3576938e1d8c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.201800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 52/104] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:59.220233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.220519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.220804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.247361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:21:59.248508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:21:59.326990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:59.334994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 53/105] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:21:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:21:59.380935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bf372-b531-48e0-a50f-3576938e1d8c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:59.404048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:59.541921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b62c7-3d6b-430f-95ef-7a1df2e3cb36 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:59.598136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:59.598800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.599084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.599360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.603272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:59.604764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:59.606424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 54/106] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:21:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:21:59.612154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:59.623540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087bf372-b531-48e0-a50f-3576938e1d8c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:59.661281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:59.661936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.662290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.662732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.674047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:59.679079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:59.679938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:21:59.680749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 53/107] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:59 2022] GET /identity/v3/roles/reader => generated 84 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:59.681391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:59.696953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:59.730585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:59.757748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966e5a98-91ad-431d-8884-de6f0cefb5c3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:59.793336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:59.793881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.794166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.794477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.794974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc968d5-6494-492e-8018-9bd39ad5a9ac None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:59.804659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:21:59.809349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:21:59.810231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:21:59.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 55/108] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:21:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:21:59.811837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:59.833225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88460e69-63a8-4176-ae26-76a0e1c19661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:59.833860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88460e69-63a8-4176-ae26-76a0e1c19661 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.834127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88460e69-63a8-4176-ae26-76a0e1c19661 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.834430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88460e69-63a8-4176-ae26-76a0e1c19661 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.835071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88460e69-63a8-4176-ae26-76a0e1c19661 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:59.836537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88460e69-63a8-4176-ae26-76a0e1c19661 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:59.863179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 54/109] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:21:59 2022] POST /identity/v3/projects => generated 302 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:21:59.864416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac11b2-59e8-481b-8c45-aeec99a0d8e1 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.864708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac11b2-59e8-481b-8c45-aeec99a0d8e1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.864984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfac11b2-59e8-481b-8c45-aeec99a0d8e1 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.866252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 55/110] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:59.867372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2cc29-503c-4fa1-b608-956048e11ea8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.867660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2cc29-503c-4fa1-b608-956048e11ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.867934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f2cc29-503c-4fa1-b608-956048e11ea8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.869210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 56/111] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:21:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:21:59.869987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2cc29-503c-4fa1-b608-956048e11ea8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:21:59.939376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b9724a-0dae-4c05-b2fc-dd228fdda03b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:21:59.948900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:59.971199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:21:59.980883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:21:59.981513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:21:59.981837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:21:59.982115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:21:59.982868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:21:59.983177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:21:59.984668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:21:59.994281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 56/112] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:21:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:21:59.995300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.001154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f2cc29-503c-4fa1-b608-956048e11ea8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.045371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540b80e2-832e-42f5-864a-4719691acf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.046365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b80e2-832e-42f5-864a-4719691acf95 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.046658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b80e2-832e-42f5-864a-4719691acf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.046933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540b80e2-832e-42f5-864a-4719691acf95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.060924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540b80e2-832e-42f5-864a-4719691acf95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-540b80e2-832e-42f5-864a-4719691acf95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:00.066308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:00.067180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:00.068013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 57/113] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:21:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:00.069267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e841e-e271-475e-b6c8-bef9a2198497 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.069267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e841e-e271-475e-b6c8-bef9a2198497 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.069267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e841e-e271-475e-b6c8-bef9a2198497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.088251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c44e841e-e271-475e-b6c8-bef9a2198497 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:00.089461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c44e841e-e271-475e-b6c8-bef9a2198497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:00.121696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d8b2b-78bf-4520-b3dd-6b8be1206b26 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.161786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.162422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.162709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.162985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.163657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:00.171726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 58/114] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:00.171892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:00.171985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.171985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.171985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.172208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:00.183504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 57/115] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:21:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:00.184436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.189776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:00.191041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:00.248515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:00.253666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:00.272919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 59/116] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:00.274072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.344081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb6291b-7acb-4adb-b377-76e364a7f12e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.406792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.407431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.407714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.407992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] PATH_INFO: `/v3/users/73bb478c9c9745f09a1d05899ac91e07` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.434079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] RBAC: Authorizing `identity:get_user(user_id=73bb478c9c9745f09a1d05899ac91e07)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:00.435614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:00.437348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 58/117] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/users/73bb478c9c9745f09a1d05899ac91e07 => generated 289 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:00.438137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.445854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645ea97b-e982-4d39-aef2-f66c940b5463 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.495764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.497180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.497469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.497745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.498444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:00.498754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:00.500173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:00.509200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 60/118] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:00.510212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.557209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:00.576806 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf34dbec-8e7b-4472-ac94-0331b7caf209 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.589173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:00.614162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.614760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.615054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.615307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.616428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:00.617868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:00.618868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2ced05-660f-44ed-9ee0-1e16ad8f019e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.619633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 59/119] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:00.624616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.658170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.658806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.659088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.659351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.731788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d81506-b4c9-4398-b406-9d7c349c8f11 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.732314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:00.733644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:00.769565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.770650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.770650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.770840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.781371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:00.781580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:00.786414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:00.787364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:00.788463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:00.792517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 60/120] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:00.793373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.836916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:00.841536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 61/121] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:00.842419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.912805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86312eba-810c-4277-aced-a3cec0e002ab None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.953713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4532bd-f269-4619-a42e-2bf4fe9d9d91 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:00.956986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.957916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.958431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:00.958721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:00.959927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:00.961415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:00.963069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 61/122] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:00.963855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:00.998126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:00.998795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:00.999603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.000142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.013220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:01.018772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:01.019747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:01.020579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 62/123] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:01.021752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.030324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.079516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d2409-ae4c-42a0-99e8-509860da415f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.088836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.121541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.122449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.122737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.123019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.124766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.127160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.128844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 62/124] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:00 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.128844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.149703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a1d94-1334-45d3-8c3e-706402d45e05 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.187473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.188014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.188298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.188578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.189246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:01.189554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.190970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.199274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 63/125] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.200085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.245683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68984dc4-0711-4114-b59c-56b1331be43f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.272322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.283657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.284304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.284924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.284924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.286002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.287493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.289299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 63/126] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.294716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.312669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee4104-a7d5-4103-aeb6-bb2384dd65dd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.342088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.349620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.350307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.350596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.350872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.352016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.353479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.355155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 64/127] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.359818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.404209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf48ba7-6e32-4628-8f91-b680cb64f4db None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.441247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.441781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.442051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.442362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.443032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:01.443358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.444726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.454680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 64/128] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.455530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.492449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd89147-ca6c-4261-92d2-ae3a564a11f7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.531666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.532331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.532611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.532883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b/users/7d825b9d979b45a095dd6e2cabfe9971/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.534801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] RBAC: Authorizing `identity:create_grant(project_id=c50530f607054a5981b1291dff6e980b, user_id=7d825b9d979b45a095dd6e2cabfe9971, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.536252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.538479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.552822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 65/129] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:01 2022] PUT /identity/v3/projects/c50530f607054a5981b1291dff6e980b/users/7d825b9d979b45a095dd6e2cabfe9971/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:01.554070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.590324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5fbe1-2b5a-455f-acbc-ef774a81f8b2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.595125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.652298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.653366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.653799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.654553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.658132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.661321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.670088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 65/130] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.670088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.708475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96c5216-7c6a-4a49-893e-bbee3807edfd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.752871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.753499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.753782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.754054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.755095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] RBAC: Adding query filter params (role.id=5ca9e6a00b28491faebc747f756cd173, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=73bb478c9c9745f09a1d05899ac91e07) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:01.755403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.757868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.764211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 66/131] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/role_assignments?user.id=73bb478c9c9745f09a1d05899ac91e07&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=5ca9e6a00b28491faebc747f756cd173 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:01.770212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.824532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ba9de3-7bb6-4a71-9e8a-f4e544348d8a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:01.893951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:01.894673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:01.894967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:01.895242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b/users/785cf2fb0f794ab8b0d643d7797ff5dc/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:01.896685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.897954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] RBAC: Authorizing `identity:create_grant(project_id=c50530f607054a5981b1291dff6e980b, user_id=785cf2fb0f794ab8b0d643d7797ff5dc, role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:01.899522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:01.922858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 66/132] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:01 2022] PUT /identity/v3/projects/c50530f607054a5981b1291dff6e980b/users/785cf2fb0f794ab8b0d643d7797ff5dc/roles/712bb84fd46d48f28cbe60cbb255213f => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:01.923919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:01.976230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:01.980234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c8d14c-023c-46b1-8fce-b0d26e99e059 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:02.029836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d8f86-a45a-4f97-a2fd-ef83d6504173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:02.032940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d8f86-a45a-4f97-a2fd-ef83d6504173 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.033412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d8f86-a45a-4f97-a2fd-ef83d6504173 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.033826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d8f86-a45a-4f97-a2fd-ef83d6504173 None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.037396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d8f86-a45a-4f97-a2fd-ef83d6504173 None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:02.039217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d8f86-a45a-4f97-a2fd-ef83d6504173 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:02.041273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 67/133] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:02.048196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b09762-3aa4-4445-b30b-a0520dd6cbb4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.049794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b09762-3aa4-4445-b30b-a0520dd6cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.052385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b09762-3aa4-4445-b30b-a0520dd6cbb4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.058660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 68/134] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:02 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:02.060410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b09762-3aa4-4445-b30b-a0520dd6cbb4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:02.149972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5ffbe8-e64c-4124-b702-9b55dd567fed None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:02.197842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81683a31-fb3d-47f7-973e-7e4f64ab6cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:02.198891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81683a31-fb3d-47f7-973e-7e4f64ab6cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.199314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81683a31-fb3d-47f7-973e-7e4f64ab6cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.200188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81683a31-fb3d-47f7-973e-7e4f64ab6cd9 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.201902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81683a31-fb3d-47f7-973e-7e4f64ab6cd9 None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:02.204676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81683a31-fb3d-47f7-973e-7e4f64ab6cd9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:02.207491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 67/135] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:01 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:02.209078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.209576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.210422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.231895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b09762-3aa4-4445-b30b-a0520dd6cbb4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:02.244840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:02.246317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:02.292053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:02.309598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:02.311078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.312526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.313031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] PATH_INFO: `/v3/projects/b930502887274f279815f7b672deeabd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.315655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] RBAC: Authorizing `identity:get_project(project_id=b930502887274f279815f7b672deeabd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:02.318025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:02.321825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 69/136] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:02 2022] GET /identity/v3/projects/b930502887274f279815f7b672deeabd => generated 312 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:02.323761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:02.395259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 68/137] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:02.396328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:02.451727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:02.518950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eff69a-88f4-4e93-a2ab-53f09ed5f3eb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:02.600427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:02.602893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.603395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.603761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b/users/785cf2fb0f794ab8b0d643d7797ff5dc/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.608791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] RBAC: Authorizing `identity:create_grant(project_id=c50530f607054a5981b1291dff6e980b, user_id=785cf2fb0f794ab8b0d643d7797ff5dc, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:02.612698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:02.620237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d451c4-33ea-4997-a63d-90da88f83e2a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:02.649378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 70/138] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:02 2022] PUT /identity/v3/projects/c50530f607054a5981b1291dff6e980b/users/785cf2fb0f794ab8b0d643d7797ff5dc/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:02.683521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:02.684335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:02.684627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:02.684907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] PATH_INFO: `/v3/projects/b930502887274f279815f7b672deeabd/users/785cf2fb0f794ab8b0d643d7797ff5dc/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:02.687153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] RBAC: Authorizing `identity:create_grant(project_id=b930502887274f279815f7b672deeabd, user_id=785cf2fb0f794ab8b0d643d7797ff5dc, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:02.688725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:02.710645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 69/139] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:02 2022] PUT /identity/v3/projects/b930502887274f279815f7b672deeabd/users/785cf2fb0f794ab8b0d643d7797ff5dc/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:02.951846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:03.017876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:03.072137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48baae8f-3592-4ad4-bc08-3450b2d9f06f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:03.122200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c17266f-1ed5-4850-b622-6cf9ed2af452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:03.122851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17266f-1ed5-4850-b622-6cf9ed2af452 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:03.123132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17266f-1ed5-4850-b622-6cf9ed2af452 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:03.123406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c17266f-1ed5-4850-b622-6cf9ed2af452 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:03.126962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17266f-1ed5-4850-b622-6cf9ed2af452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:03.128623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c17266f-1ed5-4850-b622-6cf9ed2af452 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:03.130733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 71/140] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:03.135950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:03.294663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:03.300597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24783860-0c61-43d3-9f0c-85c83d2dd362 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:03.340894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63662b74-1eff-4f01-9e7d-cf46c2d3a54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:03.341507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63662b74-1eff-4f01-9e7d-cf46c2d3a54f None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:03.341786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63662b74-1eff-4f01-9e7d-cf46c2d3a54f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:03.342069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63662b74-1eff-4f01-9e7d-cf46c2d3a54f None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:03.343756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63662b74-1eff-4f01-9e7d-cf46c2d3a54f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:03.345238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63662b74-1eff-4f01-9e7d-cf46c2d3a54f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:03.367891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 70/141] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:03 2022] POST /identity/v3/users => generated 283 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:03.898898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4affa-f4b3-4c85-90fb-6dcbc81377ef None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:03.899226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4affa-f4b3-4c85-90fb-6dcbc81377ef None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:03.899456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af4affa-f4b3-4c85-90fb-6dcbc81377ef None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:03.900861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 72/142] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:03.908724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:03.909027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:03.909287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:03.928911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:03.930121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:04.017625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 71/143] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:04.385955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4affa-f4b3-4c85-90fb-6dcbc81377ef None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:04.518451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af4affa-f4b3-4c85-90fb-6dcbc81377ef None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:04.548757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:04.562173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7ccdf-bfe6-480a-bb23-6a7e2d1e6820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:04.562840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ccdf-bfe6-480a-bb23-6a7e2d1e6820 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:04.563180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ccdf-bfe6-480a-bb23-6a7e2d1e6820 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:04.563461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7ccdf-bfe6-480a-bb23-6a7e2d1e6820 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:04.577202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7ccdf-bfe6-480a-bb23-6a7e2d1e6820 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-63b7ccdf-bfe6-480a-bb23-6a7e2d1e6820 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:04.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:04.583915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:04.585222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 73/144] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:04.590418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:04.672303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d5b1-7090-41e3-bbea-ff55a2e8ab59 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:04.672303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d5b1-7090-41e3-bbea-ff55a2e8ab59 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:04.672303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab5d5b1-7090-41e3-bbea-ff55a2e8ab59 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:04.673228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 74/145] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:04 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:04.679039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7afbe6-bd23-4d59-91c4-715bf8bdde01 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:04.679586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7afbe6-bd23-4d59-91c4-715bf8bdde01 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:04.680061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7afbe6-bd23-4d59-91c4-715bf8bdde01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:04.717171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:04.743245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-be7afbe6-bd23-4d59-91c4-715bf8bdde01 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:04.744484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-be7afbe6-bd23-4d59-91c4-715bf8bdde01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:04.774245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62906cc5-370f-421d-bdaf-1152f99c65f3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:04.839326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:04.841563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:04.842119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:04.842693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:04.844136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:04.844648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:04.846375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:04.861001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 72/146] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:04.868829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:04.970053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 75/147] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:04.975622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8406b-21cd-44a8-8982-e1dc313d2c8e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:04.975936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8406b-21cd-44a8-8982-e1dc313d2c8e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:04.976193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8406b-21cd-44a8-8982-e1dc313d2c8e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:04.977697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 76/148] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:04.995421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:04.995421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:04.995421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.036877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cd71c-dc97-46f6-a4c3-b5c59af8939c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:05.043680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:05.044899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:05.106799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:05.107455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:05.107738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:05.108011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] PATH_INFO: `/v3/users/7d825b9d979b45a095dd6e2cabfe9971` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.112537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] RBAC: Authorizing `identity:get_user(user_id=7d825b9d979b45a095dd6e2cabfe9971)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:05.114023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:05.118027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 73/149] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:04 2022] GET /identity/v3/users/7d825b9d979b45a095dd6e2cabfe9971 => generated 247 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:05.124421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:05.137849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:05.207795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 77/150] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:05.295575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2d12f-ad55-43a2-a8c3-b33c8115f677 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:05.342092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:05.342937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:05.343221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:05.343491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.434637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:05.436115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:05.450538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:05.511289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:05.548420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 74/151] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:05.554791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:05.618361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:05.668820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b43af52-7f3c-4ede-8426-c1f1c127fd68 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:05.687213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:05.705451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36b861-1df2-42f2-ab5d-d6d6ecf50db4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:05.721984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:05.722583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:05.722960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:05.723238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.726996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:05.728666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:05.730592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 75/152] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:05 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:05.737179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:05.741843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:05.742520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:05.742826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:05.743061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.754973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:05.759710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:05.760569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:05.761350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:05.765895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 78/153] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:05.767216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:05.885796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82dbd41-709f-4400-be48-04134e1d2640 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:05.887293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:05.903657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9a9f93-c3a9-4785-b191-c170ddb4d6dd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:05.951828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:05.952668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:05.952959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:05.953230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.953830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:05.953950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:05.954218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:05.954465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:05.954740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:05.955012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:05.955623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:05.955724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=7d825b9d979b45a095dd6e2cabfe9971) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:05.956029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:05.957584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:05.964381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 76/154] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:05 2022] GET /identity/v3/role_assignments?user.id=7d825b9d979b45a095dd6e2cabfe9971&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:05.973894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 79/155] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:05 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:05.979832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.008018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.048817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:06.124510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8564e0e-fc37-4a1e-ae18-780432c14c5b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.142861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71e25b-069c-4296-963d-b9a0531038e0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.162064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.163195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.163682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.164170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.183294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:06.188709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:06.189710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:06.190681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:06.194307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 80/156] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:06.196023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.226619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:06.239586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.240262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.240544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.240821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] PATH_INFO: `/v3/users/73bb478c9c9745f09a1d05899ac91e07` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.242030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] RBAC: Authorizing `identity:get_user(user_id=73bb478c9c9745f09a1d05899ac91e07)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:06.243572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:06.245225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 77/157] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:05 2022] GET /identity/v3/users/73bb478c9c9745f09a1d05899ac91e07 => generated 289 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:06.263699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.308676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349aeefb-0f55-4dfe-9f74-a37f46eb10fa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.350555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.351247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.351534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.351805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.352482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:06.352784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:06.354195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:06.368234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 81/158] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:06.373198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.509558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ef4a7-b438-41a9-b134-5812bb7ac6b6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.526562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453fd81-8c10-4183-9c71-24cf270ad02f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.531459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:06.538671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:06.547523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.548170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.548455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.548732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.549887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:06.551382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:06.552970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 82/159] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:06 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:06.558758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.581638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.582305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.582589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.582864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.584023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:06.585507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:06.587523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 78/160] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:06 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:06.596116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:06.696716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dff927-b77d-4bdc-a28e-350b3d85de93 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.738393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade57997-6ce9-47ef-8802-9adb9de00d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.739091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade57997-6ce9-47ef-8802-9adb9de00d9c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.739371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade57997-6ce9-47ef-8802-9adb9de00d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.739645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade57997-6ce9-47ef-8802-9adb9de00d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.767702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:06.850056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade57997-6ce9-47ef-8802-9adb9de00d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:06.851562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade57997-6ce9-47ef-8802-9adb9de00d9c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:06.856473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8304f0ff-5bd2-4963-881e-dad89a2afcdd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:06.924847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38896e8-6c75-49fd-9b3e-3de52243c3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:06.925519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38896e8-6c75-49fd-9b3e-3de52243c3af None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.925800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38896e8-6c75-49fd-9b3e-3de52243c3af None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.926080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38896e8-6c75-49fd-9b3e-3de52243c3af None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b/users/73bb478c9c9745f09a1d05899ac91e07/roles/5ca9e6a00b28491faebc747f756cd173` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.928968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38896e8-6c75-49fd-9b3e-3de52243c3af None admin] RBAC: Authorizing `identity:create_grant(project_id=c50530f607054a5981b1291dff6e980b, user_id=73bb478c9c9745f09a1d05899ac91e07, role_id=5ca9e6a00b28491faebc747f756cd173)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:06.934371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38896e8-6c75-49fd-9b3e-3de52243c3af None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:06.955771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 79/161] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:06 2022] PUT /identity/v3/projects/c50530f607054a5981b1291dff6e980b/users/73bb478c9c9745f09a1d05899ac91e07/roles/5ca9e6a00b28491faebc747f756cd173 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:06.958456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0852d9-5e95-48b1-9228-92a6e69030ad None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.958740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0852d9-5e95-48b1-9228-92a6e69030ad None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.959024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0852d9-5e95-48b1-9228-92a6e69030ad None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:06.964007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 80/162] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:06.971683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:06.971984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:06.972235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.008219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:07.009447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:07.073869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:07.097293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 83/163] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:22:07.098714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1909ec-5dc5-4ebf-9a8f-fa2c9e994569 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:07.098899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1909ec-5dc5-4ebf-9a8f-fa2c9e994569 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:07.100912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1909ec-5dc5-4ebf-9a8f-fa2c9e994569 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.100912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 84/164] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:07.114766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:07.115188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:07.115432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.149334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:07.150576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:07.192742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 81/165] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:07.193644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:07.232316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:07.255238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 85/166] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:07.354534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee90538-aeb4-44d7-94fb-69e4aed07fb4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:07.400135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:07.400774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:07.401054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:07.401426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.402764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:07.404242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:07.406958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 82/167] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:07 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:07.413084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:07.493094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:07.540231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b474c56-da04-44d3-b039-0a8145c53284 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:07.571324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:07.601736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:07.602395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:07.602733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:07.603095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.603944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] RBAC: Adding query filter params (role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=785cf2fb0f794ab8b0d643d7797ff5dc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:07.604255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:07.606309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:07.622737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 86/168] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:07 2022] GET /identity/v3/role_assignments?user.id=785cf2fb0f794ab8b0d643d7797ff5dc&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:07.690493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c725a06b-d68c-4beb-957b-1f7c022eaccf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:07.737601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:07.738273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:07.738548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:07.738813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.756301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:07.761093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:07.762282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:07.763710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 83/169] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:07.771367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:07.857380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:07.902002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:07.937597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f005143-26eb-4f10-90be-9de3d8673976 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:07.996331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:07.996962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:07.997240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:07.997517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:07.998229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:07.998553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:08.000105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aec7ee-db81-4167-96c5-0f50421b8df3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:08.017461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 87/170] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:08.021836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584c9a7-5b04-4f0e-9523-053cf34ae93e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.022343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584c9a7-5b04-4f0e-9523-053cf34ae93e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.022700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e584c9a7-5b04-4f0e-9523-053cf34ae93e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.024192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 88/171] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:08.024853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584c9a7-5b04-4f0e-9523-053cf34ae93e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:08.126157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:08.138520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d22da-7c1c-4023-9778-d8951bf153fe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.167776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e584c9a7-5b04-4f0e-9523-053cf34ae93e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.190571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:08.207313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad1de96-09e1-4c1c-ae83-23eacc2f301a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.207965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1de96-09e1-4c1c-ae83-23eacc2f301a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.208235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1de96-09e1-4c1c-ae83-23eacc2f301a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.208680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad1de96-09e1-4c1c-ae83-23eacc2f301a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.228099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad1de96-09e1-4c1c-ae83-23eacc2f301a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9ad1de96-09e1-4c1c-ae83-23eacc2f301a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:08.238497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:08.239628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:08.240632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:08.242042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.242868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.243322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.243744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.245601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 84/172] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:22:08.246429 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:08.248147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:08.249649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.251532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 89/173] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:08 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:08.251980 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.252404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.257246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:08.282311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:08.285124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:08.384348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 85/174] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:08.385281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:08.392208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98113134-281c-4e8c-8d6b-cb42772749ae None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.441696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.442589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.443164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.443761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.444655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:08.445124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:08.446805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fd44cf-b8ad-4185-8204-bdcdefb56b52 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:08.462044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 90/175] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:08.463300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7618-4e99-4b3b-b8d9-dea9ad787e82 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.463599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7618-4e99-4b3b-b8d9-dea9ad787e82 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.463855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336f7618-4e99-4b3b-b8d9-dea9ad787e82 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.466650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 91/176] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:08.467377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7618-4e99-4b3b-b8d9-dea9ad787e82 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:08.545302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:08.560040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:08.583913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e8fc1-4faa-4212-90fb-7aaee7cdda96 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.609609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336f7618-4e99-4b3b-b8d9-dea9ad787e82 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.628097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.629065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.629344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.629618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.647734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b608f-6800-4dfc-8865-84477ae650ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.648377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b608f-6800-4dfc-8865-84477ae650ba None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.648774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b608f-6800-4dfc-8865-84477ae650ba None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.649051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b608f-6800-4dfc-8865-84477ae650ba None admin] PATH_INFO: `/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.650347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b608f-6800-4dfc-8865-84477ae650ba None admin] RBAC: Authorizing `identity:get_user(user_id=785cf2fb0f794ab8b0d643d7797ff5dc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:08.651807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b608f-6800-4dfc-8865-84477ae650ba None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:08.653802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 92/177] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:08 2022] GET /identity/v3/users/785cf2fb0f794ab8b0d643d7797ff5dc => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:08.654771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172b6e4-3cec-484e-a187-d4247e05d12e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.655050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172b6e4-3cec-484e-a187-d4247e05d12e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.656238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172b6e4-3cec-484e-a187-d4247e05d12e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.675391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5172b6e4-3cec-484e-a187-d4247e05d12e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:08.676670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5172b6e4-3cec-484e-a187-d4247e05d12e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:08.711358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:08.714724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:08.752319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 93/178] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:08.753598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c9967-9516-417f-985d-3303b86f560b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.753887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c9967-9516-417f-985d-3303b86f560b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.754165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352c9967-9516-417f-985d-3303b86f560b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.758313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 94/179] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:08.758313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c9967-9516-417f-985d-3303b86f560b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:08.808640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:08.811696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:08.834481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 86/180] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:08.839868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:08.877707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352c9967-9516-417f-985d-3303b86f560b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.922730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.923262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.923539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.923811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.947245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8846eb-3ab0-491d-8f6e-c1df4b1ef7c8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:08.984050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd05863f-f8ab-40e0-988e-c0323e4cd446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:08.984681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05863f-f8ab-40e0-988e-c0323e4cd446 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:08.984967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05863f-f8ab-40e0-988e-c0323e4cd446 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:08.985245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd05863f-f8ab-40e0-988e-c0323e4cd446 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:08.995596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd05863f-f8ab-40e0-988e-c0323e4cd446 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-cd05863f-f8ab-40e0-988e-c0323e4cd446 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:09.000848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:09.001734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:09.002581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 87/181] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:09.003730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.003890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.004132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.011305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.013685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.029169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:09.031632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:09.061121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.076008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.118216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 88/182] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:09.126355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.132159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 95/183] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:09.133442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.245249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51cf064-8a62-4292-862f-13412a240a6b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.246157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f358803-0550-4d35-a00f-f62550339987 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.283939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.284547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.284824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.285148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.285822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:09.286121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.287646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.288459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.289291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.289575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.289850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] PATH_INFO: `/v3/projects/b930502887274f279815f7b672deeabd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.291381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] RBAC: Authorizing `identity:get_project(project_id=b930502887274f279815f7b672deeabd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.292858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.293352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.297893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 96/184] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.298763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.300717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 89/185] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/projects/b930502887274f279815f7b672deeabd => generated 312 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.302228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.361765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.425306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa04579-e3ff-43f2-abd4-f15cf96427af None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.428199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeb367c-e38b-43e4-9a13-35db38ef32ae None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.464556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.464764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.465185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.465461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.465732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.466010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.466465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.466877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.467065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.468356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.469976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 90/186] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.482828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:09.488921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:09.489815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:09.490662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 97/187] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:09.491340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.560628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.595053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7afac2-48b9-48de-9324-3121aafd4242 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.605224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.609424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8b19e-c63b-4d3c-998b-afd95f5362f7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.634163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.635327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.635811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.636281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.637473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=b930502887274f279815f7b672deeabd, user.id=785cf2fb0f794ab8b0d643d7797ff5dc) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:09.637926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.639815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.644988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.645626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.645909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.646209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] PATH_INFO: `/v3/users/d6df8b1c64cd44beb465f00e2478f959` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.647483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 91/188] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/role_assignments?user.id=785cf2fb0f794ab8b0d643d7797ff5dc&scope.project.id=b930502887274f279815f7b672deeabd&role.id=d89a893a140749d39b9cecdedae58d78 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.656228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.664278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] RBAC: Authorizing `identity:get_user(user_id=d6df8b1c64cd44beb465f00e2478f959)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.665888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.667830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 98/189] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/users/d6df8b1c64cd44beb465f00e2478f959 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.668825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.783255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6e73a9-9bb6-45d4-9e09-d7183de92d5b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.786263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e107b1-ae46-4faa-bbff-596b7c949704 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.808789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.832073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.832938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.833356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.833782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.834747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=785cf2fb0f794ab8b0d643d7797ff5dc) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:09.834875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.836312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.836794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:09.837438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:09.837720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:09.837996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:09.839180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:09.840674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:09.842329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 92/190] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.843645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.848311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 99/191] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/role_assignments?user.id=785cf2fb0f794ab8b0d643d7797ff5dc&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=d89a893a140749d39b9cecdedae58d78 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:09.849414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:09.909276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:09.985907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8dae79b-b491-4a62-8bd6-d932e1dd26a6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:09.987087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945171bd-c993-430f-b1ed-d8221dc64dba None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.028897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.029515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.029796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.029900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.030067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.031778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.032269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.032724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.033865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:10.034494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.036937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29158723-057e-43a8-a5b9-89f932dd15f3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.053389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 93/192] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:10.055029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe43e8-9c3e-49b1-814c-dcf238222955 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.055407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe43e8-9c3e-49b1-814c-dcf238222955 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.055559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafe43e8-9c3e-49b1-814c-dcf238222955 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.056835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 94/193] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:10.057537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe43e8-9c3e-49b1-814c-dcf238222955 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.090269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.112623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.114130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.172450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafe43e8-9c3e-49b1-814c-dcf238222955 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.174168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.207899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c4436-75be-4edb-bc4e-b834a62ed6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.208504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c4436-75be-4edb-bc4e-b834a62ed6aa None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.208781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c4436-75be-4edb-bc4e-b834a62ed6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.209051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c4436-75be-4edb-bc4e-b834a62ed6aa None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.209701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c4436-75be-4edb-bc4e-b834a62ed6aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.211194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522c4436-75be-4edb-bc4e-b834a62ed6aa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.215670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 100/194] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:10.217029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.233763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 95/195] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:10 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:10.235190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.235474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.235748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.267966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:10.269841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:10.317672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.340443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a20343-bfe6-483e-8465-fb3ebfb2e363 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.349132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 96/196] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:10.349821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.392379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.393013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.393285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.393555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] PATH_INFO: `/v3/users/d6df8b1c64cd44beb465f00e2478f959` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.395135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] RBAC: Authorizing `identity:get_user(user_id=d6df8b1c64cd44beb465f00e2478f959)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.396783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.398634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 101/197] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:10 2022] GET /identity/v3/users/d6df8b1c64cd44beb465f00e2478f959 => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:10.406290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.468417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.484268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1740e6c5-1bed-4c2d-979a-e6c962e395b3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.523540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.524183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.524458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.524726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.526110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.527771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.529945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 97/198] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:10 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:10.536704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.542737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d106cbb-0ab5-4e7f-9a86-404ee5b6e8fa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.587946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.589329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.590381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.590978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.614818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.667512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008f191-1989-41b5-a89c-101dfddc1b60 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.677512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.679325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.707267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.708003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.708407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.708692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:10.709579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=d6df8b1c64cd44beb465f00e2478f959) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:10.709886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:10.712006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:10.720364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 98/199] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:10 2022] GET /identity/v3/role_assignments?user.id=d6df8b1c64cd44beb465f00e2478f959&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:10.721241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.769548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.862141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 102/200] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:22:10.868553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:10.911920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175afe30-7b1b-4b84-ad32-c82b2a9afecf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:10.932258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:10.982235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:10.983317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:10.983787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:10.984208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:11.001975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:11.008101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:11.009041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:11.010427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 99/201] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:10 2022] GET /identity/v3/roles/reader => generated 84 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:11.011281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d63d5bb-7829-4996-bd7d-489692a77afc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:11.016088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:11.057272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:11.058303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:11.060163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:11.060641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:11.063109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:11.068083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:11.072013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 103/202] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:10 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:11.075970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:11.166567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:11.171408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0da735-fa57-4ab4-8b9d-9974660fc899 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:11.213466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:11.214499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:11.215262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:11.216467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:11.217628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:11.218966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:11.220747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:11.236582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 100/203] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:11.242594 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:11.328028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d95b3c-1c63-4ee2-b962-9f400d65e650 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:11.374930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:11.388725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:11.389726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:11.391343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:11.397246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:11.398228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] RBAC: Adding query filter params (role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=d6df8b1c64cd44beb465f00e2478f959) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:11.398748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:11.400393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:11.410309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 104/204] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:11 2022] GET /identity/v3/role_assignments?user.id=d6df8b1c64cd44beb465f00e2478f959&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 259 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:11.435731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f6a868-17cd-406c-971a-a12ec2a0a18a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:11.478862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:11.479862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:11.480368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:11.480835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] PATH_INFO: `/v3/users/73bb478c9c9745f09a1d05899ac91e07` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:11.486932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] RBAC: Authorizing `identity:get_user(user_id=73bb478c9c9745f09a1d05899ac91e07)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:11.488931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:11.490877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 101/205] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:11 2022] GET /identity/v3/users/73bb478c9c9745f09a1d05899ac91e07 => generated 289 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:11.497059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:11.633484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a57967-878a-47fb-a76b-44998523205e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:11.686343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:11.687882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:11.688365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:11.688848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:11.759215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:11.778764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:11.780319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:11.938401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 105/206] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:11.944265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:11.963718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:12.089401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531b4ba-1e7f-4ba9-a8f1-025745c421b2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:12.159245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc56f6ad-5d84-43ce-8977-a71e1f06f17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:12.160251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56f6ad-5d84-43ce-8977-a71e1f06f17c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:12.160710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56f6ad-5d84-43ce-8977-a71e1f06f17c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:12.161156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc56f6ad-5d84-43ce-8977-a71e1f06f17c None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:12.163081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56f6ad-5d84-43ce-8977-a71e1f06f17c None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:12.165433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc56f6ad-5d84-43ce-8977-a71e1f06f17c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:12.167952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 102/207] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:11 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:12.175957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:12.348058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:12.375529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0c42c2-4f7d-4e37-b2d6-cb24e59cd18f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:12.429360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:12.429981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:12.430313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:12.430559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:12.431269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] RBAC: Adding query filter params (role.id=5ca9e6a00b28491faebc747f756cd173, scope.project.id=c50530f607054a5981b1291dff6e980b, user.id=73bb478c9c9745f09a1d05899ac91e07) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:12.431572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:12.433022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9c83e1-359e-4765-8a85-f9f27d05ce85 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:12.442859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 106/208] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:12 2022] GET /identity/v3/role_assignments?user.id=73bb478c9c9745f09a1d05899ac91e07&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=5ca9e6a00b28491faebc747f756cd173 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:12.833084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353ae06-0833-4c73-aae2-1ede41a47a9a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:12.833998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353ae06-0833-4c73-aae2-1ede41a47a9a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:12.834810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353ae06-0833-4c73-aae2-1ede41a47a9a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:12.837406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 103/209] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:12.847943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376f263-1375-47d1-9a00-5241895c9aa5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:12.849382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376f263-1375-47d1-9a00-5241895c9aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:12.850159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0376f263-1375-47d1-9a00-5241895c9aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:12.875972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0376f263-1375-47d1-9a00-5241895c9aa5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:12.877548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0376f263-1375-47d1-9a00-5241895c9aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:12.974860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 107/210] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:12.989744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf000ff-7ede-4bad-aebc-d9941fe641cc None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:12.990070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf000ff-7ede-4bad-aebc-d9941fe641cc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:12.990349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf000ff-7ede-4bad-aebc-d9941fe641cc None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:12.991785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 104/211] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:12.999151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b5dbe-b36b-4305-9068-2f5f612cc13b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:12.999828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b5dbe-b36b-4305-9068-2f5f612cc13b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.000309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b5dbe-b36b-4305-9068-2f5f612cc13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.022616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ed5b5dbe-b36b-4305-9068-2f5f612cc13b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:13.024156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ed5b5dbe-b36b-4305-9068-2f5f612cc13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:13.113703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 108/212] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:13.229948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18169ca4-a80f-4b63-8da0-51f1e0de1dc3 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.230506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18169ca4-a80f-4b63-8da0-51f1e0de1dc3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.230506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18169ca4-a80f-4b63-8da0-51f1e0de1dc3 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.231910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 105/213] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:13.238161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.238761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.239237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.252640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:13.258678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:13.259860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:13.299896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18169ca4-a80f-4b63-8da0-51f1e0de1dc3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:13.335019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 109/214] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:13.429349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:13.460816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:13.467594 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18169ca4-a80f-4b63-8da0-51f1e0de1dc3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:13.525379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:13.526397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.526849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.527374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.546622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:13.556353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:13.558054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:13.559621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:13.564564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 106/215] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:22:13.567899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:13.635880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac396a5d-ade7-467b-b85a-a0188840735b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:13.673752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:13.674407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.674692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.674960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.676150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:13.677588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:13.679332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 110/216] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:13.683155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:13.686879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:13.700955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9741eb70-9c55-4961-81cd-cd3caba0d8f4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:13.750091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:13.751165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.751165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.751524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.752232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:13.752618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:13.754096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:13.762532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:13.773789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 107/217] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:13.774720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:13.808987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf8bce-4b86-4840-93c8-6792042ccaa1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:13.850286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:13.850920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.851192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.851492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.862612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:13.867404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:13.869110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 111/218] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:13.870796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:13.941990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e32b62-629a-45e3-9e85-529f0c38e159 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:13.954295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:13.984149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:13.984896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:13.985245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:13.985589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:13.986592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:13.988578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.006855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52fa28c-da7b-4770-aa6e-a4e8d0091098 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.011409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 108/219] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:13 2022] POST /identity/v3/users => generated 297 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:14.012283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.054743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972dd6b7-fa02-4a79-a697-d4e596548bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.055507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dd6b7-fa02-4a79-a697-d4e596548bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.055788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dd6b7-fa02-4a79-a697-d4e596548bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.056068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972dd6b7-fa02-4a79-a697-d4e596548bd1 None admin] PATH_INFO: `/v3/users/4c2e574b2c2b4becbeff4e7de87c6b3d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.076244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dd6b7-fa02-4a79-a697-d4e596548bd1 None admin] RBAC: Authorizing `identity:get_user(user_id=4c2e574b2c2b4becbeff4e7de87c6b3d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.078397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972dd6b7-fa02-4a79-a697-d4e596548bd1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.080505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 112/220] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:13 2022] GET /identity/v3/users/4c2e574b2c2b4becbeff4e7de87c6b3d => generated 297 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:14.082939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc0cf-da85-4724-8912-e85a1a4e6fdc None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.082939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc0cf-da85-4724-8912-e85a1a4e6fdc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.082939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecc0cf-da85-4724-8912-e85a1a4e6fdc None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.090850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 113/221] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:14.090850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc0cf-da85-4724-8912-e85a1a4e6fdc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.118988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:14.183733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1946afe4-fbeb-4b8b-bcf5-e585a66f841d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.231760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.232377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.232623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.232869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.234232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:14.235148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.236592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac228e-766f-4451-bf87-270b37fe8794 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.245136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 109/222] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:14.247829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecc0cf-da85-4724-8912-e85a1a4e6fdc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.247951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.247951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.248106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.268614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:14.269842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:14.280084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:14.295160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.296824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.297303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.297788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.361665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 110/223] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:14.362591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.384549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.386341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.448348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:14.534958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 114/224] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:14.541151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.543992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acbe56c-07b8-49c2-a8f8-fc1e475c5753 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.566124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:14.585000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.585921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.586424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.586867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] PATH_INFO: `/v3/users/d6df8b1c64cd44beb465f00e2478f959` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.588220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] RBAC: Authorizing `identity:get_user(user_id=d6df8b1c64cd44beb465f00e2478f959)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.589942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.591836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 111/225] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/users/d6df8b1c64cd44beb465f00e2478f959 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:14.597379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.712436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6e549f-91a9-4d70-a2d3-49163f9e2cb9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.766472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.767289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.767476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.767748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.768917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.770407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.772067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 115/226] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:14.772847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.772974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a489ded5-95b6-4e18-9613-4dfeeb320d7d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.798731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:14.817466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.818092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.818393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.818663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.820379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.822345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.825075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 112/227] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:14.825868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:14.882022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:14.898462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f925e848-7a96-4783-a71d-fe3b53bb61bf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.953082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf1226-e989-4da6-a558-bca877955b1c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:14.953980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.955187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.955877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.956477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.972830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:14.982625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:14.984902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:14.986252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:14.988948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 116/228] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:14.994368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:14.994368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:14.994368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:14.994368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:14.994984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=4c2e574b2c2b4becbeff4e7de87c6b3d) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:14.994984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:14.995235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:14.996806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:15.001366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 113/229] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:14 2022] GET /identity/v3/role_assignments?user.id=4c2e574b2c2b4becbeff4e7de87c6b3d&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=d89a893a140749d39b9cecdedae58d78 => generated 259 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:15.002169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:15.141065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:15.218781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcf4ec9-700c-425a-a04c-8a1d8de46206 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:15.234646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe68a991-3e7d-4e28-b7b0-1a5ef19fa4f0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:15.262353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:15.263902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:15.264530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:15.265064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/d6df8b1c64cd44beb465f00e2478f959/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:15.267385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] RBAC: Authorizing `identity:create_grant(project_id=15406dd00e9a441da00290b8a3dcb066, user_id=d6df8b1c64cd44beb465f00e2478f959, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:15.269196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:15.286246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 117/230] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:14 2022] PUT /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/d6df8b1c64cd44beb465f00e2478f959/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:15.310863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:15.311490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:15.311827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:15.312100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:15.312778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:15.313078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:15.314490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:15.330850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 114/231] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:15.337217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:15.354282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:15.460792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c020149-d61e-4686-9947-03383f642ff1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:15.500292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40842881-302e-476e-9c5e-446367065c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:15.501325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40842881-302e-476e-9c5e-446367065c12 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:15.501803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40842881-302e-476e-9c5e-446367065c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:15.502305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40842881-302e-476e-9c5e-446367065c12 None admin] PATH_INFO: `/v3/users/d6df8b1c64cd44beb465f00e2478f959` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:15.503689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40842881-302e-476e-9c5e-446367065c12 None admin] RBAC: Authorizing `identity:get_user(user_id=d6df8b1c64cd44beb465f00e2478f959)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:15.505383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40842881-302e-476e-9c5e-446367065c12 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:15.507518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 118/232] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:15 2022] GET /identity/v3/users/d6df8b1c64cd44beb465f00e2478f959 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:15.513274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:15.631506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b321c54f-0334-4b59-ab1c-2e51df4a53d9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:15.650573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:15.668952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe4d494-9266-4266-be91-9b5f8cdb86b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:15.669583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4d494-9266-4266-be91-9b5f8cdb86b7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:15.669860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4d494-9266-4266-be91-9b5f8cdb86b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:15.670130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe4d494-9266-4266-be91-9b5f8cdb86b7 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:15.671327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4d494-9266-4266-be91-9b5f8cdb86b7 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:15.672790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe4d494-9266-4266-be91-9b5f8cdb86b7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:15.674826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 115/233] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:15 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:15.680565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40842881-302e-476e-9c5e-446367065c12 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:15.765948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:15.790030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40842881-302e-476e-9c5e-446367065c12 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:15.825879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7d00a-668f-4bc6-ac8a-f2e16e127ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:15.826581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d00a-668f-4bc6-ac8a-f2e16e127ed3 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:15.826856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d00a-668f-4bc6-ac8a-f2e16e127ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:15.827126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7d00a-668f-4bc6-ac8a-f2e16e127ed3 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/d6df8b1c64cd44beb465f00e2478f959/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:15.829887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7d00a-668f-4bc6-ac8a-f2e16e127ed3 None admin] RBAC: Authorizing `identity:create_grant(project_id=15406dd00e9a441da00290b8a3dcb066, user_id=d6df8b1c64cd44beb465f00e2478f959, role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:15.831373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7d00a-668f-4bc6-ac8a-f2e16e127ed3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:15.849018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 119/234] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:15 2022] PUT /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/d6df8b1c64cd44beb465f00e2478f959/roles/712bb84fd46d48f28cbe60cbb255213f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:16.441747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631b21-0c91-4f90-9fee-68c21186af5d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:16.442609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631b21-0c91-4f90-9fee-68c21186af5d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:16.442609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17631b21-0c91-4f90-9fee-68c21186af5d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:16.443858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 116/235] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:16.445193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4e13c-75ba-4a9e-9162-fc39c0ea167f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:16.445766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4e13c-75ba-4a9e-9162-fc39c0ea167f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:16.445766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4e13c-75ba-4a9e-9162-fc39c0ea167f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:16.447332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 120/236] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:16.452556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:16.452868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:16.453142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:16.459150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:16.459430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:16.459701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:16.484668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:16.485874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:16.498460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:16.499685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:16.578258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 121/237] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:16.620439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 117/238] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:16.983316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:17.013587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:17.072604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:17.128605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75682e0f-7921-4ae1-8858-ff3e3be8f6f0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:17.205460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:17.206107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.206403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.206788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.212769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:17.214553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:17.214809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86201328-7f6e-4ee2-a730-5c18b8312e8a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:17.216538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 122/239] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:17.223629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:17.242179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:17.282488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:17.283252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.283531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.283805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.298359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:17.303073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:17.304102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:17.305117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 118/240] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:17.315426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:17.396154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f17b71-2b74-4d32-a3b8-7517430afbeb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:17.438724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4d1eb-1e90-4d5c-8f9e-d5dcb5c4085b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:17.439350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4d1eb-1e90-4d5c-8f9e-d5dcb5c4085b None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.439629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4d1eb-1e90-4d5c-8f9e-d5dcb5c4085b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.439902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4d1eb-1e90-4d5c-8f9e-d5dcb5c4085b None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.440563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4d1eb-1e90-4d5c-8f9e-d5dcb5c4085b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:17.441998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4d1eb-1e90-4d5c-8f9e-d5dcb5c4085b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:17.462111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:17.483255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 123/241] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:17 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:22:17.485724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073ffd-17d3-4107-b0a2-2515f616834a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.486231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073ffd-17d3-4107-b0a2-2515f616834a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.486506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51073ffd-17d3-4107-b0a2-2515f616834a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.489139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 124/242] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:17.494973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.495188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.495495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.511234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e8ebb-d71b-4dd5-be95-fd9dfa8c6efc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:17.517497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:17.519639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:17.560345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:17.561249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.561540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.561964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.562804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:17.563121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:17.564781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:17.576490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 119/243] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:17.582377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:17.615229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 125/244] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:17.646807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:17.737765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4a175-955e-424d-86d6-5233f4a4e7ad None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:17.775028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:17.775625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.775905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.776175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] PATH_INFO: `/v3/users/8dde9ad7c6cf463287833813d622ac08` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.792548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=8dde9ad7c6cf463287833813d622ac08)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:17.794456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:17.796424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 120/245] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:17 2022] GET /identity/v3/users/8dde9ad7c6cf463287833813d622ac08 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:17.802247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:17.919592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43b6478-7b1f-41d1-b669-b46626ba1174 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:17.949834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:17.964943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:17.965964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:17.966494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:17.966970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:17.985209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.064711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:18.066269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:18.112031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:18.129049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473391fd-e12c-4776-8e6c-0e42f556f7c3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:18.171842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:18.172493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.172771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.173043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.184439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:18.185538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 126/246] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:18.189777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:18.190781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:18.191641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 121/247] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:18.193569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.211609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.226682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:18.345659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a9ab60-1e55-424d-b23f-cd26a60bd476 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:18.348533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6089b95-170e-4022-8414-d479af2fe0b8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:18.393303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:18.393957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.394322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.394668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.395420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:18.395791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:18.397305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:18.397693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7c6af-9f90-45c3-a60f-d827d606797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:18.399010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7c6af-9f90-45c3-a60f-d827d606797d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.399714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7c6af-9f90-45c3-a60f-d827d606797d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.400396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7c6af-9f90-45c3-a60f-d827d606797d None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.402155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7c6af-9f90-45c3-a60f-d827d606797d None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:18.404049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7c6af-9f90-45c3-a60f-d827d606797d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:18.407230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 127/248] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:18 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:18.408164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 122/249] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:18.409487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837efb3-490b-44ff-8dd1-3d64405fd3c5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.410242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837efb3-490b-44ff-8dd1-3d64405fd3c5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.412847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.415064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9837efb3-490b-44ff-8dd1-3d64405fd3c5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.416500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 128/250] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:18.417286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837efb3-490b-44ff-8dd1-3d64405fd3c5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.431037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:18.538813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:18.541864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b504de-da17-4f5e-8efe-b52a6ab13aa1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:18.551585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9837efb3-490b-44ff-8dd1-3d64405fd3c5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:18.581930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:18.582928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.583220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.583595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.584437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] RBAC: Adding query filter params (role.id=5ca9e6a00b28491faebc747f756cd173, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=8dde9ad7c6cf463287833813d622ac08) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:18.584850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:18.586765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1620769d-d737-45fb-b219-3511bbb08602 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:18.594427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 123/251] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:18 2022] GET /identity/v3/role_assignments?user.id=8dde9ad7c6cf463287833813d622ac08&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=5ca9e6a00b28491faebc747f756cd173 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:18.595114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b788ff-022d-49bc-bf40-b2bbee8b6146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:18.595952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2248776-84c7-4d21-931b-17c7e8a514bc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.596179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b788ff-022d-49bc-bf40-b2bbee8b6146 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.596402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2248776-84c7-4d21-931b-17c7e8a514bc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.596740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b788ff-022d-49bc-bf40-b2bbee8b6146 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.596862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2248776-84c7-4d21-931b-17c7e8a514bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.597254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b788ff-022d-49bc-bf40-b2bbee8b6146 None admin] PATH_INFO: `/v3/users/4c2e574b2c2b4becbeff4e7de87c6b3d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.606328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b788ff-022d-49bc-bf40-b2bbee8b6146 None admin] RBAC: Authorizing `identity:get_user(user_id=4c2e574b2c2b4becbeff4e7de87c6b3d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:18.606328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b788ff-022d-49bc-bf40-b2bbee8b6146 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:18.614640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 129/252] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:18 2022] GET /identity/v3/users/4c2e574b2c2b4becbeff4e7de87c6b3d => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:18.620851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bb2d1-54f3-4527-beaf-843e687f90aa None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.621158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bb2d1-54f3-4527-beaf-843e687f90aa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.621412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063bb2d1-54f3-4527-beaf-843e687f90aa None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.622903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 130/253] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:18.623699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bb2d1-54f3-4527-beaf-843e687f90aa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.636825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c2248776-84c7-4d21-931b-17c7e8a514bc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:18.638070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c2248776-84c7-4d21-931b-17c7e8a514bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:18.763947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 124/254] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:18.765753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.766036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.766340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.786227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:18.799382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:18.800888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:18.810128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063bb2d1-54f3-4527-beaf-843e687f90aa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:18.875495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:18.876062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:18.876333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:18.876607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:18.883017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:18.884633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:18.891055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 131/255] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:18 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:18.899364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:18.919395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 125/256] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:18.950607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:19.046233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60d8f4-836a-4e01-a207-9fd60fa781fd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:19.096869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:19.097599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:19.097885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:19.098160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/4c2e574b2c2b4becbeff4e7de87c6b3d/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:19.100135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] RBAC: Authorizing `identity:create_grant(project_id=15406dd00e9a441da00290b8a3dcb066, user_id=4c2e574b2c2b4becbeff4e7de87c6b3d, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:19.101594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:19.126790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 132/257] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:18 2022] PUT /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/4c2e574b2c2b4becbeff4e7de87c6b3d/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:22:19.242999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:19.302695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:19.399817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc697b-16c3-4024-9e98-c70485c8d0c4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:19.438933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:19.439573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:19.439849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:19.440123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:19.442104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:19.451795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:19.457058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:19.458017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:19.458859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:19.459476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 126/258] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:19.467520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:19.471225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:19.508412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065a630-0c20-4af8-b8ef-c925a96f453d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:19.574087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:19.574861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:19.575284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:19.575558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:19.602161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:19.613877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:19.614884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:19.615691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:19.618218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 133/259] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:19.620795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:19.676202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee938a4-ef26-452e-be57-fffe67bee2d7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:19.750281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:19.751124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:19.751403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:19.752344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:19.753017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:19.753317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:19.754744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:19.777713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 127/260] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:19.785102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:19.825622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0105c03-6ebf-4041-86e4-d2f079410e97 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:19.896323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:19.896944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:19.897219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:19.897487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:19.898157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:19.898618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:19.900293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:19.916628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 134/261] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:19.918633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:19.927243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:19.941846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:20.049412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a92c36-5125-4955-9a92-b769817d75fe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:20.135883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedfc79-0242-47aa-bc9c-f3efa6f14e26 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:20.147848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:20.148505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.148789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.149069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] PATH_INFO: `/v3/users/d6df8b1c64cd44beb465f00e2478f959` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.150317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] RBAC: Authorizing `identity:get_user(user_id=d6df8b1c64cd44beb465f00e2478f959)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:20.151796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:20.154656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 128/262] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:19 2022] GET /identity/v3/users/d6df8b1c64cd44beb465f00e2478f959 => generated 283 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:20.163205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:20.174341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:20.174985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.175260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.175526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] PATH_INFO: `/v3/users/d6df8b1c64cd44beb465f00e2478f959` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.176667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] RBAC: Authorizing `identity:get_user(user_id=d6df8b1c64cd44beb465f00e2478f959)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:20.178212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:20.180235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 135/263] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:19 2022] GET /identity/v3/users/d6df8b1c64cd44beb465f00e2478f959 => generated 283 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:20.196354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:20.326574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:20.361409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065da3a-a83b-4c42-809a-3ea4a83d4006 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:20.371051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e5e9a-0ec6-4495-8536-d7b7284f5429 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:20.412689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:20.426423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c37ad10-6def-433d-93e2-21945e5c6c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:20.427152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37ad10-6def-433d-93e2-21945e5c6c8b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.427433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37ad10-6def-433d-93e2-21945e5c6c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.427749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c37ad10-6def-433d-93e2-21945e5c6c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.435902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877b767-ede4-454a-ab82-a7809f14f669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:20.436464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877b767-ede4-454a-ab82-a7809f14f669 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.437283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877b767-ede4-454a-ab82-a7809f14f669 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.437559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877b767-ede4-454a-ab82-a7809f14f669 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.559147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37ad10-6def-433d-93e2-21945e5c6c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:20.560670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c37ad10-6def-433d-93e2-21945e5c6c8b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:20.563161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877b767-ede4-454a-ab82-a7809f14f669 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:20.564645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877b767-ede4-454a-ab82-a7809f14f669 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:20.723187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:20.726252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 129/264] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 563 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:20.728368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931bb55-f5a9-4073-950a-d88256f6d889 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.729126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931bb55-f5a9-4073-950a-d88256f6d889 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.729632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e931bb55-f5a9-4073-950a-d88256f6d889 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.731500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 130/265] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:20.732871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515613b-14a9-4997-802d-4b26d356b1ed None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.733422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515613b-14a9-4997-802d-4b26d356b1ed None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.734466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515613b-14a9-4997-802d-4b26d356b1ed None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.736271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 131/266] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:20.741173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515613b-14a9-4997-802d-4b26d356b1ed None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:20.762208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 136/267] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:20.763740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cb4aa-1f4f-4714-a75d-93905ec1d71d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.764187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cb4aa-1f4f-4714-a75d-93905ec1d71d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.764600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280cb4aa-1f4f-4714-a75d-93905ec1d71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.776998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:20.793356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-280cb4aa-1f4f-4714-a75d-93905ec1d71d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:20.795187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-280cb4aa-1f4f-4714-a75d-93905ec1d71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:20.877153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515613b-14a9-4997-802d-4b26d356b1ed None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:20.886649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 137/268] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:20.888617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.888978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.889324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.912227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:20.913731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:20.930982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:20.931628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:20.931907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:20.932177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:20.933325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:20.934829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:20.943520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 132/269] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:20 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:20.944487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.060226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 138/270] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:21.061082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.118672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.148795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.168278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba926e-40c2-450f-9f22-664c6706e820 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.215045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f20faa-8b88-41dd-85a8-9c2822901ba4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.247745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.248388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.248662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.248928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.250081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:21.251578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:21.256131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 133/271] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:20 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:21.264335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.285658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.286292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.286566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.286834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.287622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] RBAC: Adding query filter params (role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=d6df8b1c64cd44beb465f00e2478f959) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:21.287922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:21.290551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:21.302526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 139/272] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:21 2022] GET /identity/v3/role_assignments?user.id=d6df8b1c64cd44beb465f00e2478f959&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 608 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:21.385138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.425294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530445f2-8d9e-407b-9854-8f669f3d982b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.495179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.496246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.496738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.497211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.498525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=d6df8b1c64cd44beb465f00e2478f959) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:21.498959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:21.501169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b701a19-ed32-421a-8a05-2cb98199eb89 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:21.525098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 134/273] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:21 2022] GET /identity/v3/role_assignments?user.id=d6df8b1c64cd44beb465f00e2478f959&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:21.566768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999a7ab-7336-45a1-9524-7c0d08c33d9a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.567100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999a7ab-7336-45a1-9524-7c0d08c33d9a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.567316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999a7ab-7336-45a1-9524-7c0d08c33d9a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.570518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 135/274] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:21.574829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf03f3-2a22-4621-863f-ac7688fe2036 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.575370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.575652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.575922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.578685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.623367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:21.624644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:21.636282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.637183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.637716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.638208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.640841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:21.644278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:21.650567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 140/275] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:21.651991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.662649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.744309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 136/276] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:21.745319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.807395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932dad8-1524-4f03-8bac-02faac9f281a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.849819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.850890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.851490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.851967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.864945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:21.870884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:21.871824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:21.872637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:21.875954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 141/277] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:21.883151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:21.896026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24889f6a-b2c2-469f-9f7a-0502a5599073 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:21.930966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:21.960492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc459b21-cdd5-479c-85fe-85b5b26fc25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:21.961567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459b21-cdd5-479c-85fe-85b5b26fc25e None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:21.962057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459b21-cdd5-479c-85fe-85b5b26fc25e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:21.962563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc459b21-cdd5-479c-85fe-85b5b26fc25e None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:21.963486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc459b21-cdd5-479c-85fe-85b5b26fc25e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:21.965196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc459b21-cdd5-479c-85fe-85b5b26fc25e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:21.991969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 137/278] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:21 2022] POST /identity/v3/users => generated 293 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:22.030956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad3f91-811d-4fe3-a4aa-f1a621fb7bfc None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.031282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad3f91-811d-4fe3-a4aa-f1a621fb7bfc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.031768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddad3f91-811d-4fe3-a4aa-f1a621fb7bfc None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.033168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 138/279] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:22.039374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.039714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.039914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.061088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:22.063007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:22.067391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:22.074564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1750b4-9fa9-494e-94e5-75b2f682005d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.145915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.146976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.147190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.147461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.148146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:22.148447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:22.149866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:22.162046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 142/280] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:22.162985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.185609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 139/281] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:22.188047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.288373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:22.299974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c185a-9ddf-4b0f-93b8-2a904d82eac0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.339855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadefbc3-bd05-4013-8a43-77710d956479 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.357315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.358228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.359184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.359548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.375200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:22.381122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:22.382163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:22.383155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 143/282] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:22.390222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.391126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.391317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.391607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.392043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] PATH_INFO: `/v3/users/0eb5503e09ad469aa25bae44c2691da2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.412668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:22.422104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] RBAC: Authorizing `identity:get_user(user_id=0eb5503e09ad469aa25bae44c2691da2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:22.423978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:22.426248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 140/283] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/users/0eb5503e09ad469aa25bae44c2691da2 => generated 293 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:22.431909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.550894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67c50e1-1989-4134-a2aa-dd4dc01e9092 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.587652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f37a5c-14d5-49e1-9b36-11ef1a827434 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.591906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.592739 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.593758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.594212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.595097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:22.595516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:22.597328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:22.609146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 144/284] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:22.610329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.629102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:22.647555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.648765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.649298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.649758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.651274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, user.id=0eb5503e09ad469aa25bae44c2691da2, scope.system=all) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:22.651847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:22.654377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:22.664223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 141/285] 10.222.0.91 () {56 vars in 1308 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/role_assignments?user.id=0eb5503e09ad469aa25bae44c2691da2&scope.system=all&role.id=d89a893a140749d39b9cecdedae58d78 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:22.665642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.743284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f193a64-880b-4dba-9fac-4678de496adc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.790383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.791161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.791563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.791908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.805631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:22.811101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:22.813762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:22.815784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 145/286] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:22.822212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:22.822607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:22.908630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e263f1-5468-441f-99f9-610a7a8780e3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.924501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:22.971894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0975c84-4d99-49d6-b701-632eae500b86 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:22.988090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:22.988737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:22.989019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:22.989291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] PATH_INFO: `/v3/users/8dde9ad7c6cf463287833813d622ac08` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:22.990597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] RBAC: Authorizing `identity:get_user(user_id=8dde9ad7c6cf463287833813d622ac08)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:22.992208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:22.999661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 142/287] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/users/8dde9ad7c6cf463287833813d622ac08 => generated 297 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:23.000924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:23.017291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:23.018246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:23.018701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:23.019143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:23.020099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:23.020406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:23.021897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:23.032449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 146/288] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:23.037871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:23.231728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571f1c4-77ac-4445-b529-a67781c6aed2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:23.275145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7195982-a2f4-4c77-aaa8-812223b3c224 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:23.290949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:23.304153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:23.304795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:23.305075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:23.305351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:23.307365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:23.308835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:23.313358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 143/289] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:22 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:23.319957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:23.350751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:23.351401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:23.351678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:23.351955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] PATH_INFO: `/v3/users/4c2e574b2c2b4becbeff4e7de87c6b3d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:23.354282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] RBAC: Authorizing `identity:get_user(user_id=4c2e574b2c2b4becbeff4e7de87c6b3d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:23.359387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:23.363574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 147/290] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:23 2022] GET /identity/v3/users/4c2e574b2c2b4becbeff4e7de87c6b3d => generated 297 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:23.370068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:23.373508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:23.472327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716f92e-9f97-4ef4-b3d3-c97b7aa767dd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:23.517158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:23.519028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:23.520173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:23.520801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/8dde9ad7c6cf463287833813d622ac08/roles/5ca9e6a00b28491faebc747f756cd173` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:23.525381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] RBAC: Authorizing `identity:create_grant(project_id=15406dd00e9a441da00290b8a3dcb066, user_id=8dde9ad7c6cf463287833813d622ac08, role_id=5ca9e6a00b28491faebc747f756cd173)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:23.527425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:23.557121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 144/291] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:22:23 2022] PUT /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066/users/8dde9ad7c6cf463287833813d622ac08/roles/5ca9e6a00b28491faebc747f756cd173 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:23.592001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc110a51-02db-462e-ab3a-da7583599f85 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:23.648271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:23.649823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:23.650580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:23.651304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:23.762296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:23.764229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:23.767438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:23.914701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 148/292] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:22:23.923221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:23.991940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:24.043251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ce18c1-8387-4e83-99b2-b7f1e415656c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:24.086013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f168683-ec15-425a-8cfc-32b6894cd3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:24.087150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f168683-ec15-425a-8cfc-32b6894cd3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.087846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f168683-ec15-425a-8cfc-32b6894cd3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.088369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f168683-ec15-425a-8cfc-32b6894cd3c6 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.089748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f168683-ec15-425a-8cfc-32b6894cd3c6 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:24.091448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f168683-ec15-425a-8cfc-32b6894cd3c6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:24.093836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 145/293] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:23 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:24.100190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:24.223335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbd7f-eaac-440d-b78a-9c50158884dc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:24.224942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:24.271704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:24.272319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.272597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.272864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.273590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=4c2e574b2c2b4becbeff4e7de87c6b3d) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:24.273898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:24.275353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1ffb66-4ad0-45dc-98ab-745d4a3bd6b0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:24.283033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 149/294] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:24 2022] GET /identity/v3/role_assignments?user.id=4c2e574b2c2b4becbeff4e7de87c6b3d&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=d89a893a140749d39b9cecdedae58d78 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:24.509354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398396b3-dda7-49b0-b0fc-05102778710a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.509770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398396b3-dda7-49b0-b0fc-05102778710a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.510071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398396b3-dda7-49b0-b0fc-05102778710a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.511859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 146/295] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:24.518231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64632-dbb3-42d6-b81a-3b8de43faa51 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.518518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64632-dbb3-42d6-b81a-3b8de43faa51 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.518785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb64632-dbb3-42d6-b81a-3b8de43faa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.542525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8eb64632-dbb3-42d6-b81a-3b8de43faa51 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:24.543842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8eb64632-dbb3-42d6-b81a-3b8de43faa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:24.625075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 150/296] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:24.929545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398396b3-dda7-49b0-b0fc-05102778710a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:24.987615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5e35-e387-45dd-8ff5-4d8a34ce033f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:24.988131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5e35-e387-45dd-8ff5-4d8a34ce033f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:24.988131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519e5e35-e387-45dd-8ff5-4d8a34ce033f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:24.989904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 151/297] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:25.002392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05c319-b3f0-4b8d-a603-48235ee7f827 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.003020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05c319-b3f0-4b8d-a603-48235ee7f827 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.003340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e05c319-b3f0-4b8d-a603-48235ee7f827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.032574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7e05c319-b3f0-4b8d-a603-48235ee7f827 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:25.034144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7e05c319-b3f0-4b8d-a603-48235ee7f827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:25.065938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398396b3-dda7-49b0-b0fc-05102778710a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.094065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:25.106096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea470e3a-6c0e-4466-b365-953396f54083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.106096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea470e3a-6c0e-4466-b365-953396f54083 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.106096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea470e3a-6c0e-4466-b365-953396f54083 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.106096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea470e3a-6c0e-4466-b365-953396f54083 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.118080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea470e3a-6c0e-4466-b365-953396f54083 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ea470e3a-6c0e-4466-b365-953396f54083 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:25.123214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:25.124842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:25.125725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 147/298] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:25.126751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9f1ac-e324-4879-875f-7769d88ae32f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.126997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9f1ac-e324-4879-875f-7769d88ae32f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.127334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f9f1ac-e324-4879-875f-7769d88ae32f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.129468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 148/299] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:25.130555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9f1ac-e324-4879-875f-7769d88ae32f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.150211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 152/300] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:25.150211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.150211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.150211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.174317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:25.175902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:25.213615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:25.269868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 153/301] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:25.287496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f9f1ac-e324-4879-875f-7769d88ae32f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.329554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.330166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.330477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.330798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.331470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:25.331774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:25.333289 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:25.343081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 149/302] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:25.348669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.428028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.479314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81efb26-6e63-4cc6-b82d-9bddffb59409 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.486578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:25.527037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.527985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.528499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.528979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] PATH_INFO: `/v3/users/4b3c35a0010e455a81e1ba8a60969975` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.547021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] RBAC: Authorizing `identity:get_user(user_id=4b3c35a0010e455a81e1ba8a60969975)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:25.549732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:25.551374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c9cce0-12ab-4959-af25-800ff112e609 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.552524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 154/303] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:25 2022] GET /identity/v3/users/4b3c35a0010e455a81e1ba8a60969975 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:25.560010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.578411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:25.596147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.597182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.597630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.598003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.599469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:25.601343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:25.607789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 150/304] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:25.607789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.720575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a7abf-c9e8-4964-936a-888a2378f9dc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.758131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6258d-9ce7-4707-8380-f35bd7a89973 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.764936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.766235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.766652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.767066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.768224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] RBAC: Adding query filter params (role.id=5ca9e6a00b28491faebc747f756cd173, user.id=4b3c35a0010e455a81e1ba8a60969975, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:25.768631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:25.770172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:25.776933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 155/305] 10.222.0.91 () {56 vars in 1308 bytes} [Wed Aug 24 03:22:25 2022] GET /identity/v3/role_assignments?user.id=4b3c35a0010e455a81e1ba8a60969975&scope.system=all&role.id=5ca9e6a00b28491faebc747f756cd173 => generated 226 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:25.776933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.792942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:25.830053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.831927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.832543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.832924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.851820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:25.862972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:25.863984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:25.864778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 151/306] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:25.872192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:25.905107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:25.938973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c32453-d03e-42ac-ae97-b1122237105d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:25.988367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:25.989032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:25.989314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:25.989590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] PATH_INFO: `/v3/groups` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:25.990631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:25.992329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:26.017008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a135f3ef-a6d4-47b3-8051-b963ba075397 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:26.022234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 156/307] 10.222.0.91 () {60 vars in 1134 bytes} [Wed Aug 24 03:22:25 2022] POST /identity/v3/groups => generated 229 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:26.064176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:26.065202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.065688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.066155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.067101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:26.067645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:26.069286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d89f629-475e-42bd-a14a-2ed4a4643e8a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:26.080036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 152/308] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:26.088278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:26.091558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e7ac2-02c4-44c3-9e43-3e1c8e8bd301 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.091865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e7ac2-02c4-44c3-9e43-3e1c8e8bd301 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.092109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e7ac2-02c4-44c3-9e43-3e1c8e8bd301 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.097955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 153/309] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:26.099940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.100299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.100643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.125332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:26.126780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:26.160407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:26.206781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 154/310] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:26.270614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a1d237-d9d5-4fb6-b35c-4d6680cd7f27 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:26.314990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:26.316007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.316466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.316897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] PATH_INFO: `/v3/users/0eb5503e09ad469aa25bae44c2691da2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.318800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] RBAC: Authorizing `identity:get_user(user_id=0eb5503e09ad469aa25bae44c2691da2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:26.321357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:26.324785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 157/311] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:26 2022] GET /identity/v3/users/0eb5503e09ad469aa25bae44c2691da2 => generated 293 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:26.333984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:26.469669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd41da5-ea9f-4566-8a27-18c9621d2b60 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:26.493979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:26.507648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:26.525437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:26.526375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.526792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.527213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] PATH_INFO: `/v3/system/users/0eb5503e09ad469aa25bae44c2691da2/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.528887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=0eb5503e09ad469aa25bae44c2691da2, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:26.531296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:26.547554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 155/312] 10.222.0.91 () {58 vars in 1334 bytes} [Wed Aug 24 03:22:26 2022] PUT /identity/v3/system/users/0eb5503e09ad469aa25bae44c2691da2/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:26.680412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4274c-bde9-4d90-be74-a85d588b38c8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:26.720831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:26.721541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.721820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.722094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.736088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:26.741721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:26.742747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:26.743818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 158/313] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:26.751183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:26.787630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:26.864239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c341f0-324d-4090-b6eb-164a68eb4d33 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:26.907617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:26.908238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:26.908514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:26.908785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:26.909459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:26.909757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:26.911172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:26.921242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 156/314] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:26.926903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:27.024036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:27.062415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba5561-907a-436f-b7a3-eea49c5dc200 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:27.112251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:27.113244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.113723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.114295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] PATH_INFO: `/v3/users/8dde9ad7c6cf463287833813d622ac08` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.115995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] RBAC: Authorizing `identity:get_user(user_id=8dde9ad7c6cf463287833813d622ac08)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:27.117820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:27.120003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 159/315] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:26 2022] GET /identity/v3/users/8dde9ad7c6cf463287833813d622ac08 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:27.127916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:27.229552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:27.237200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0c316c-3181-408c-895f-9b033c863725 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:27.274742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:27.275503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.275905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.276335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.355644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:27.357239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:27.457342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:27.464520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 157/316] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:27.470668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:27.607169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9b4b13-6cbf-4814-b436-1afaa08ce567 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:27.645431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f715b00-a11f-433c-baef-f6a62851cb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:27.646500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715b00-a11f-433c-baef-f6a62851cb0c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.646985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715b00-a11f-433c-baef-f6a62851cb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.647457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f715b00-a11f-433c-baef-f6a62851cb0c None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.648947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f715b00-a11f-433c-baef-f6a62851cb0c None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:27.651123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f715b00-a11f-433c-baef-f6a62851cb0c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:27.653815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 160/317] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:27 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:27.659593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:27.780173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c075d-f491-4b88-b883-5aeb619e4d43 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:27.835307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:27.836395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.836867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.837339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.838776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] RBAC: Adding query filter params (role.id=5ca9e6a00b28491faebc747f756cd173, scope.project.id=15406dd00e9a441da00290b8a3dcb066, user.id=8dde9ad7c6cf463287833813d622ac08) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:27.839395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:27.842200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbb7b21-fc3d-453f-855d-c7b20c87ad87 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:27.854440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 158/318] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:27 2022] GET /identity/v3/role_assignments?user.id=8dde9ad7c6cf463287833813d622ac08&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=5ca9e6a00b28491faebc747f756cd173 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:27.964607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a401531-bc18-4761-8fa6-2d2156a36a8e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.965370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a401531-bc18-4761-8fa6-2d2156a36a8e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.965870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a401531-bc18-4761-8fa6-2d2156a36a8e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.968483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 161/319] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:27.974568 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add776b-2932-4c2f-b4d5-8829b7aaec98 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:27.975257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add776b-2932-4c2f-b4d5-8829b7aaec98 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:27.975730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0add776b-2932-4c2f-b4d5-8829b7aaec98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:27.996638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0add776b-2932-4c2f-b4d5-8829b7aaec98 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:27.998290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0add776b-2932-4c2f-b4d5-8829b7aaec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:28.071663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:28.094318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 159/320] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:28.285730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9adaa-14aa-4c32-9935-0d8630b2d6ec None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.286250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9adaa-14aa-4c32-9935-0d8630b2d6ec None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.286250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb9adaa-14aa-4c32-9935-0d8630b2d6ec None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.287563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 162/321] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:28.293998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104e04-d49a-4a3e-a195-848902fdde50 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.294498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104e04-d49a-4a3e-a195-848902fdde50 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.294601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c104e04-d49a-4a3e-a195-848902fdde50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.315435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7c104e04-d49a-4a3e-a195-848902fdde50 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:28.316664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c104e04-d49a-4a3e-a195-848902fdde50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:28.398781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 160/322] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:28.404749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9adaa-14aa-4c32-9935-0d8630b2d6ec None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:28.409115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:28.549359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb9adaa-14aa-4c32-9935-0d8630b2d6ec None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:28.587948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:28.588581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.588848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.589126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.600553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:28.605909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:28.611304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:28.612146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 163/323] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:28.615400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470fe1a-2605-4524-871c-53b04f1dabbe None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.616056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470fe1a-2605-4524-871c-53b04f1dabbe None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.616343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470fe1a-2605-4524-871c-53b04f1dabbe None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.617195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:28.618091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 161/324] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:28.625783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.626325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.626741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.645102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:28.646555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:28.691789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:28.734086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 162/325] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:28.736465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:28.744221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9fa968-c932-4af1-a6fc-cff88070f924 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:28.792677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:28.793484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.793837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.794206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.795029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:28.795372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:28.796863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:28.818082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 164/326] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:28.820929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:28.828887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:28.892908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a00ab74-f4f8-4a49-a123-940f16aace4a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:28.940431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:28.941073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:28.941353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:28.941620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:28.942807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:28.944253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:28.946019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 163/327] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:28.951514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:28.993195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9445d04f-99f2-42e1-87e5-fc5346ff5cd7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.029935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:29.044358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.045288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.045725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.046682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] PATH_INFO: `/v3/users/4b3c35a0010e455a81e1ba8a60969975` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.048073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3c35a0010e455a81e1ba8a60969975)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:29.050046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:29.051833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 165/328] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:28 2022] GET /identity/v3/users/4b3c35a0010e455a81e1ba8a60969975 => generated 293 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:29.052674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:29.091333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0f3a69-a28a-46b5-abd3-8128401acf1a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.116025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:29.140382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.140991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.141274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.141546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] PATH_INFO: `/v3/groups` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.142221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:29.143775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:29.161148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 164/329] 10.222.0.91 () {60 vars in 1134 bytes} [Wed Aug 24 03:22:28 2022] POST /identity/v3/groups => generated 226 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:22:29.161967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:29.183526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510b3165-11c8-4e87-a436-a8c878b998e6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.243090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.244448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.244733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.245126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.261047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:29.266605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:29.267665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:29.268642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 166/330] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:29.274809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:29.319352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbf160c-fae6-439d-b126-a981ab4ee99a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.362029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:29.385328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.386774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.387229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.387677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] PATH_INFO: `/v3/system/users/4b3c35a0010e455a81e1ba8a60969975/roles/5ca9e6a00b28491faebc747f756cd173` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.390088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=4b3c35a0010e455a81e1ba8a60969975, role_id=5ca9e6a00b28491faebc747f756cd173)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:29.392524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:29.410361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 165/331] 10.222.0.91 () {58 vars in 1334 bytes} [Wed Aug 24 03:22:29 2022] PUT /identity/v3/system/users/4b3c35a0010e455a81e1ba8a60969975/roles/5ca9e6a00b28491faebc747f756cd173 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:29.433556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bfb584-ce52-47b8-a19f-0e8d43377229 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.492293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.493555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.493555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.493555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.494159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:29.494488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:29.495917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:29.507977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 167/332] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:29.513575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:29.573563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:29.651016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddbceba-3b41-42fe-b51b-42c2d243aa02 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.699245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1844fe-8d91-4965-b465-3cd68d75b153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.700253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1844fe-8d91-4965-b465-3cd68d75b153 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.700689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1844fe-8d91-4965-b465-3cd68d75b153 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.701144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1844fe-8d91-4965-b465-3cd68d75b153 None admin] PATH_INFO: `/v3/users/0eb5503e09ad469aa25bae44c2691da2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.705142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1844fe-8d91-4965-b465-3cd68d75b153 None admin] RBAC: Authorizing `identity:get_user(user_id=0eb5503e09ad469aa25bae44c2691da2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:29.707723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1844fe-8d91-4965-b465-3cd68d75b153 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:29.710809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 166/333] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:29 2022] GET /identity/v3/users/0eb5503e09ad469aa25bae44c2691da2 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:29.720197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:29.874304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c0b4-90ed-4aed-b4db-7521b6c2dce1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:29.904679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:29.945814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:29.946867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:29.947349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:29.947829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:29.949174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] RBAC: Adding query filter params (role.id=d89a893a140749d39b9cecdedae58d78, user.id=0eb5503e09ad469aa25bae44c2691da2, scope.system=all) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:29.949712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:29.953046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd38f6e9-7692-48b1-af1e-867ca0704d2a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:29.982310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 168/334] 10.222.0.91 () {56 vars in 1308 bytes} [Wed Aug 24 03:22:29 2022] GET /identity/v3/role_assignments?user.id=0eb5503e09ad469aa25bae44c2691da2&scope.system=all&role.id=d89a893a140749d39b9cecdedae58d78 => generated 510 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:31.777063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431991a-b084-4f5a-b9f4-39c550173bfa None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:31.777577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431991a-b084-4f5a-b9f4-39c550173bfa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:31.777983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431991a-b084-4f5a-b9f4-39c550173bfa None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:31.781383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 167/335] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:31.787869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089317-0a44-421b-a793-6cd0913dfe9c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:31.788781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089317-0a44-421b-a793-6cd0913dfe9c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:31.789260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d089317-0a44-421b-a793-6cd0913dfe9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:31.813281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0d089317-0a44-421b-a793-6cd0913dfe9c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:31.814948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0d089317-0a44-421b-a793-6cd0913dfe9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:31.898312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 169/336] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:32.114695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a5e87-d250-49d0-b512-b3b64ec769f9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:32.115179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a5e87-d250-49d0-b512-b3b64ec769f9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:32.115650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a5e87-d250-49d0-b512-b3b64ec769f9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:32.118535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 168/337] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:32.126664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79b8f-8e84-4327-999c-f5c9f3122026 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:32.126958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79b8f-8e84-4327-999c-f5c9f3122026 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:32.127218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa79b8f-8e84-4327-999c-f5c9f3122026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:32.154438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1fa79b8f-8e84-4327-999c-f5c9f3122026 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:32.155658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1fa79b8f-8e84-4327-999c-f5c9f3122026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:32.245590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 170/338] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:32.367305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d244f30-db33-482e-8853-189268e571d4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:32.367674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d244f30-db33-482e-8853-189268e571d4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:32.368139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d244f30-db33-482e-8853-189268e571d4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:32.370527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 169/339] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:32.379359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:32.379811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:32.380256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:32.420176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:32.422799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:32.461449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:32.521923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 171/340] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:32.678692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d244f30-db33-482e-8853-189268e571d4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:32.764417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:32.838269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d244f30-db33-482e-8853-189268e571d4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:32.858691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:32.922626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:32.924042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:32.924732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7106d44-7cac-4665-ac67-1aa48847c55b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:32.925499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:32.926365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:32.963394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:32.963929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:32.964202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:32.964462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:32.964643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:32.978216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:32.979272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:32.981096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:32.982850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:32.984247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:32.986015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:32.987753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 172/341] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:22:32.991496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:32.991693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 170/342] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:32.998004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:33.071829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:33.206351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9e3f7-19e3-42a9-8746-5586b05df7ff None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:33.244870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:33.245581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.245975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.246395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.247309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:33.247765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:33.249390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c990da5-0132-42a1-986b-0ff3eed4dccd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:33.254167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ec5475-eb2e-4895-b24e-241652b72de8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:33.258897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 171/343] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:33.263647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4c9f-871f-4b95-8f67-1128153952e8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.263983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4c9f-871f-4b95-8f67-1128153952e8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.264372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a4c9f-871f-4b95-8f67-1128153952e8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.265930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 172/344] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:33 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:33.266564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4c9f-871f-4b95-8f67-1128153952e8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:33.302576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:33.326750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:33.328318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.328798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.329272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.330594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:33.331169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:33.333947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcaebb9-18c5-4b29-b86a-2f79a299b88e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:33.357147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 173/345] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:33.359515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.361729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.362447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.391858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a4c9f-871f-4b95-8f67-1128153952e8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:33.397949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:33.400112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:33.433170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:33.433844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.434178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.434536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.449598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:33.459888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:33.460865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:33.462123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 173/346] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:33.463100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:33.552394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 174/347] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:33.554071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:33.588480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:33.645634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:33.657443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a81264-787e-435d-a7aa-ae1d6ec859ee None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:33.739057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ac97c-64b6-4a41-b401-36a3fe259811 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:33.743982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:33.744822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.745319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.745889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.779547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:33.780207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:33.780489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:33.780848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:33.871163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:33.872753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:33.899126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.000454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.004340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.020377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 175/348] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:34.021310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.171778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.174770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5acc9ab-d965-4da0-b4a2-a808b4dc8ad4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.197014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.200616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 174/349] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 738 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:34.207014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.227924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.228926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.229414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.230133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.231681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:34.232501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.234451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.250282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 176/350] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:34.250282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.344711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f8c44e-3603-4483-9557-6162c73a9909 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.412702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.413341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.413610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.413911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.426824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:34.434396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:34.436195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 175/351] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:34.438442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.450991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba26bfa9-faf3-41f8-bddd-927a12c12a0b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.503470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.516255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.516530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.516787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.517065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.518231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.519671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.521327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 177/352] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:34.522100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.572456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e846950-2740-476f-9a4b-882f81a430a3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.584033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.641257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8121238f-0cc6-474d-8165-97d201a59f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.642053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121238f-0cc6-474d-8165-97d201a59f7c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.642710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121238f-0cc6-474d-8165-97d201a59f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.643358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8121238f-0cc6-474d-8165-97d201a59f7c None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.645206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121238f-0cc6-474d-8165-97d201a59f7c None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.646756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8121238f-0cc6-474d-8165-97d201a59f7c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.648951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 176/353] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:34.649965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ab5e3-8317-416a-aa23-6f3e86447510 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.650532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ab5e3-8317-416a-aa23-6f3e86447510 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.650809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101ab5e3-8317-416a-aa23-6f3e86447510 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.652151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 177/354] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:34.652874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ab5e3-8317-416a-aa23-6f3e86447510 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.684659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e560b0-4a39-4894-a7af-776d12e534ca None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.738364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.739155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.739436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.739710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] PATH_INFO: `/v3/users/4b3c35a0010e455a81e1ba8a60969975` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.741970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] RBAC: Authorizing `identity:get_user(user_id=4b3c35a0010e455a81e1ba8a60969975)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.743331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.745238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 178/355] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/users/4b3c35a0010e455a81e1ba8a60969975 => generated 293 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:34.746148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.821681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101ab5e3-8317-416a-aa23-6f3e86447510 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.846038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.868558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.869256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.869532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.869801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.871232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:34.871232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.872250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.884052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 178/356] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:34.884378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:34.929540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92adaea3-0b51-482d-89de-93e9821ccbe9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:34.947521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:34.976847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81f344f-f040-417a-9c6e-6d4c47e0520d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:34.977904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f344f-f040-417a-9c6e-6d4c47e0520d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.978444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f344f-f040-417a-9c6e-6d4c47e0520d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.978912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81f344f-f040-417a-9c6e-6d4c47e0520d None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:34.990350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f344f-f040-417a-9c6e-6d4c47e0520d None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:34.991884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81f344f-f040-417a-9c6e-6d4c47e0520d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:34.993784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 179/357] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:34.994958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6130000-6d2f-4b5d-81cd-1c1151538aed None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:34.995182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6130000-6d2f-4b5d-81cd-1c1151538aed None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:34.995439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6130000-6d2f-4b5d-81cd-1c1151538aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.028896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b6130000-6d2f-4b5d-81cd-1c1151538aed None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:35.031284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b6130000-6d2f-4b5d-81cd-1c1151538aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:35.045856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8372d1-19d8-48b4-b11e-f992ddb5f398 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:35.108949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:35.109749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.110072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.110453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.112173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:35.114329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:35.117254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 179/358] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:34 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:35.118961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:35.123839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 180/359] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:35.125456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097ea0-d2a7-4c63-a719-029323f53582 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.125741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097ea0-d2a7-4c63-a719-029323f53582 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.126287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89097ea0-d2a7-4c63-a719-029323f53582 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.128647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 181/360] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:35.129393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097ea0-d2a7-4c63-a719-029323f53582 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:35.196174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:35.286454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce39d10-3de7-44a9-afd0-e19e5db871ea None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:35.318246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89097ea0-d2a7-4c63-a719-029323f53582 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:35.330148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:35.341023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:35.341641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.341920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.342211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.342955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] RBAC: Adding query filter params (role.id=5ca9e6a00b28491faebc747f756cd173, user.id=4b3c35a0010e455a81e1ba8a60969975, scope.system=all) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:35.343465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:35.344786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:35.367367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 180/361] 10.222.0.91 () {56 vars in 1308 bytes} [Wed Aug 24 03:22:35 2022] GET /identity/v3/role_assignments?user.id=4b3c35a0010e455a81e1ba8a60969975&scope.system=all&role.id=5ca9e6a00b28491faebc747f756cd173 => generated 510 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:35.368713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:35.372407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b3278b-949b-4c9c-a0e1-5834be7a50a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:35.373348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3278b-949b-4c9c-a0e1-5834be7a50a2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.373826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3278b-949b-4c9c-a0e1-5834be7a50a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.375111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b3278b-949b-4c9c-a0e1-5834be7a50a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.488574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3278b-949b-4c9c-a0e1-5834be7a50a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:35.490146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b3278b-949b-4c9c-a0e1-5834be7a50a2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:35.550074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929f030c-1a09-48ff-966e-81c5c463c975 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:35.558651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:35.599738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:35.600796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.600796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.601072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.601808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=c50530f607054a5981b1291dff6e980b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:35.602115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:35.603670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:35.611392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 181/362] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:35 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=d89a893a140749d39b9cecdedae58d78 => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:35.612081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:35.670133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 182/363] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:35.672371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.672814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.673094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.698079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:35.706314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:35.708454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:35.787560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c067b30-095f-4f29-a2e6-8b0d4838cff9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:35.841149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 183/364] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:35.842600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:35.858616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:35.860113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:35.860734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:35.861416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:35.862620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=c50530f607054a5981b1291dff6e980b) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:35.863259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:35.866045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:35.876359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 182/365] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:35 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 260 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:35.878725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:35.961191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:36.055105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e782fd-cf1c-4349-b8f6-5e66152af7fc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.126952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb2db82-46fc-4a0e-a47e-67d62f546dd1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.138078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.138773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.139101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.139363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.145352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:36.166004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:36.183951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:36.187359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:36.188360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:36.192157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 184/366] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:36.201035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:36.227607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.228550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.228875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.229319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.235230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:36.238873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:36.243463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 183/367] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:35 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:36.250835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:36.360660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cd81a1-02f9-4503-8568-7757bf39a0d0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.373849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d13e93-0914-485c-83e9-1044de56a1a4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.375893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:36.406539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.407788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.408250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.408665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.409670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:36.410208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:36.411963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:36.416137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.416996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.417477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.417940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.419566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:36.421232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:36.422920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 185/368] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:36.423665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 184/369] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:36.428450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:36.429284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:36.522773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:36.549055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90e0c95-aa5b-46f9-8b59-692b3fb5ba3c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.599909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.600896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.601189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.601471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.603388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ba43ef-cf44-4a63-b275-339a1cd89658 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.649340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.650483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.650816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.651143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.673509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:36.681911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:36.692228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:36.693189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:36.694516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 186/370] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:36.695175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:36.731834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:36.733558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:36.888384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca777d5-1cca-4a32-ba06-82987fc35c61 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:36.914898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:36.945962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:36.946671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:36.947167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:36.947525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:36.948288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=15406dd00e9a441da00290b8a3dcb066) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:36.948655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:36.950637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:36.962860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 185/371] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:22:36.964538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:36.965623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 187/372] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=d89a893a140749d39b9cecdedae58d78 => generated 260 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:36.972038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:37.141716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:37.225407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778920d9-02ec-4485-b3c5-88be0064dab8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:37.241567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6a9300-3e73-4d81-9b1e-6caa23b37e5e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:37.304176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:37.305091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.305392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.305810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.306670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:37.310213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:37.310213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:37.333797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 186/373] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:37.340625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:37.370160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:37.371172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.371486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.371869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.373789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:37.377083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:37.380471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 188/374] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:36 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:37.387092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:37.548560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:37.645767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af3b65-d88e-46fc-ac8f-4f776094cc4a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:37.647479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9dce5a-76c3-4c67-a413-e99c550b2610 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:37.685974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:37.686679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.687510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.687795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.689271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:37.690794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:37.692971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 189/375] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:37 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:37.700312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:37.730027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:37.738598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:37.739438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:37.739935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:37.740240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:37.912809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:37.916207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:37.946972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8488da06-cdd4-48e9-8898-5f6aa8f030dc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:38.007519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:38.043558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:38.044176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.044456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.044724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.045463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=15406dd00e9a441da00290b8a3dcb066) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:38.045769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:38.047886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:38.064186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 190/376] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:37 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 260 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:38.146338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 187/377] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 807 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:38.155075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:38.302556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39911d88-d5c3-4a83-ad5d-54833a5c1732 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:38.370228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:38.371107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.371468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.371821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] PATH_INFO: `/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.373403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] RBAC: Authorizing `identity:get_project(project_id=24a8303d4cc94c8ebcfdb44c0be112a5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:38.375508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:38.378966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 191/378] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:38 2022] GET /identity/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:38.383996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:38.446322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:38.562270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a063df-4bf8-40f0-8f4c-29b64b278273 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:38.616925 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38112a7b-2c4c-459f-9a9d-620276fa80c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:38.617573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112a7b-2c4c-459f-9a9d-620276fa80c6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.618277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112a7b-2c4c-459f-9a9d-620276fa80c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.618560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38112a7b-2c4c-459f-9a9d-620276fa80c6 None admin] PATH_INFO: `/v3/groups/f0c59f00fc9a41ec8715aa498a478878` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.626506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38112a7b-2c4c-459f-9a9d-620276fa80c6 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c59f00fc9a41ec8715aa498a478878)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:38.628000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38112a7b-2c4c-459f-9a9d-620276fa80c6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:38.630159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 188/379] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:38 2022] GET /identity/v3/groups/f0c59f00fc9a41ec8715aa498a478878 => generated 229 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:38.638062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:38.715374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54520ca-0692-46cd-8e97-d28732cfee72 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.716155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54520ca-0692-46cd-8e97-d28732cfee72 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.716663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54520ca-0692-46cd-8e97-d28732cfee72 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.719452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 189/380] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:38.725152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50549aea-d965-4676-95a7-d4ab020f4466 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.725697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50549aea-d965-4676-95a7-d4ab020f4466 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.726201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50549aea-d965-4676-95a7-d4ab020f4466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.761177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-50549aea-d965-4676-95a7-d4ab020f4466 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:38.763420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-50549aea-d965-4676-95a7-d4ab020f4466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:38.770394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:38.852049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 190/381] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:38.854029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc73381-b5b3-4836-ac57-103e88c2958c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.854354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc73381-b5b3-4836-ac57-103e88c2958c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.854629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc73381-b5b3-4836-ac57-103e88c2958c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.856071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 191/382] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:38.865091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.865356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.865671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.882684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b2c12c-cfa0-4e54-910b-d872cf7d685e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:38.902222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:38.902485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:38.926550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:38.971975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:38.973097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:38.973676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:38.974316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:38.975657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] RBAC: Adding query filter params (group.id=f0c59f00fc9a41ec8715aa498a478878, role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=24a8303d4cc94c8ebcfdb44c0be112a5) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:38.976297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:38.986287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:38.993316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 192/383] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:38 2022] GET /identity/v3/role_assignments?group.id=f0c59f00fc9a41ec8715aa498a478878&scope.project.id=24a8303d4cc94c8ebcfdb44c0be112a5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 260 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:39.005311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 192/384] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:39.437868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:39.530226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:39.675271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda41bdd-e516-45fa-8d5e-ae62d76321a8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:39.686549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95a7f0-5a86-4797-8315-f28c35a32329 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:39.703233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:39.743680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:39.745049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:39.746629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:39.747207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:39.747693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:39.748559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:39.750201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:39.750887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:39.773276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:39.774232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:39.781455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:39.782719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:39.783598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:39.788420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:39.789707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:39.790565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:39.794421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 193/385] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:39.794869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 193/386] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:39.800947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:39.802530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:39.836598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:39.977127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b914d0-857c-43cb-9e77-23aef77a4040 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.009872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6e4bef-073e-4356-936c-a8eeee830e8c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.017227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.017990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.018524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.018954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.019641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] RBAC: Adding query filter params (name=member) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:40.019943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.021337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa675c1a-d3b6-4241-80dc-d400194b9859 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.036438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 194/387] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:40.036438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf629-9588-4b95-84fb-17b8c2534fd7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.036438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf629-9588-4b95-84fb-17b8c2534fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.036438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf629-9588-4b95-84fb-17b8c2534fd7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.036438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 195/388] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:40.037593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caf629-9588-4b95-84fb-17b8c2534fd7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.063746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.064575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.064841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.065076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.065706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:40.066001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.067824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a22376-4acf-4170-8e1c-35745989c3f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.082308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 194/389] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:40.083651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.084013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.084211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.113676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:40.114931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:40.154528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.163389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.190674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caf629-9588-4b95-84fb-17b8c2534fd7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.216464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 195/390] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:40.217976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.250875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.251867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.252619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.253102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.254770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.258138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.261728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 196/391] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:40 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:40.268229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.355414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1208ef-7020-42ff-a1e5-78356497c4cc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.402116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.403215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.403690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.404218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.409085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.410916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.413263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 196/392] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:40 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:40.418776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.449407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af5267-cca2-47ca-a1ea-49f2e81abb15 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.458223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.539899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11914229-2d47-42c9-8e65-677338d0fc17 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.554948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.555623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.555911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.556153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.557479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.559056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.563503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 197/393] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:40 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:40.576490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.594994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.594994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.594994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.594994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.596307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.596438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.598154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 197/394] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:40 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:40.610582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.625861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.726006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba08b44-ef13-4207-a910-e65f30ce6f68 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.727654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:40.735743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72d6ac3-f0af-4ea7-a1ee-331d104409fc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:40.779538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.780202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.780472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.781275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b/groups/ca9afdcec7a541d98c499528f44c2849/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.783724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c50530f607054a5981b1291dff6e980b, group_id=ca9afdcec7a541d98c499528f44c2849, role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.785621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.787418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a0679-e641-44ea-b214-5e23b1d8b23c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:40.788356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a0679-e641-44ea-b214-5e23b1d8b23c None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:40.788641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a0679-e641-44ea-b214-5e23b1d8b23c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:40.789401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a0679-e641-44ea-b214-5e23b1d8b23c None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b/groups/ca9afdcec7a541d98c499528f44c2849/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:40.791909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a0679-e641-44ea-b214-5e23b1d8b23c None admin] RBAC: Authorizing `identity:create_grant(project_id=c50530f607054a5981b1291dff6e980b, group_id=ca9afdcec7a541d98c499528f44c2849, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:40.793284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a0679-e641-44ea-b214-5e23b1d8b23c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:40.801417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 198/395] 10.222.0.91 () {58 vars in 1442 bytes} [Wed Aug 24 03:22:40 2022] PUT /identity/v3/projects/c50530f607054a5981b1291dff6e980b/groups/ca9afdcec7a541d98c499528f44c2849/roles/712bb84fd46d48f28cbe60cbb255213f => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:40.803968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:40.833783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 198/396] 10.222.0.91 () {58 vars in 1442 bytes} [Wed Aug 24 03:22:40 2022] PUT /identity/v3/projects/c50530f607054a5981b1291dff6e980b/groups/ca9afdcec7a541d98c499528f44c2849/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:22:41.004203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21f21bb-a310-4684-9a8a-2311aaec48e4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:41.024339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a406a1-5e28-4811-8796-ac861413847e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.024693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a406a1-5e28-4811-8796-ac861413847e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.024858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a406a1-5e28-4811-8796-ac861413847e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.026998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 199/397] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:41.038831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.039698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.040421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.053481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:41.054088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.054444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.054719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.074760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:41.083186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:41.084186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:41.085102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 199/398] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:41.086631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:41.090041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:41.095274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:41.136943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:41.186584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 200/399] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:41.238135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d1cc4f-4eb3-4c7b-96da-4eeb5be92802 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:41.291853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:41.292467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.292744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.293015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.293681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:41.293980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:41.295390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:41.305458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 200/400] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:41.311514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:41.315716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:41.458427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a817a8-87d3-45aa-988a-3f98267d118d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:41.498167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e72e295-05f4-42e3-91bc-85afda2894f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:41.498832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72e295-05f4-42e3-91bc-85afda2894f5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.499108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72e295-05f4-42e3-91bc-85afda2894f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.499380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e72e295-05f4-42e3-91bc-85afda2894f5 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.500530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72e295-05f4-42e3-91bc-85afda2894f5 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:41.501998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e72e295-05f4-42e3-91bc-85afda2894f5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:41.504340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 201/401] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:41 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:41.513534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:41.523098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47ed47-e454-4294-9ced-ea63d5f80268 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.523405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47ed47-e454-4294-9ced-ea63d5f80268 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.523643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d47ed47-e454-4294-9ced-ea63d5f80268 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.525688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 202/402] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:22:41.532027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.532311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.532834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.554231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:41.555537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:41.653131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 203/403] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:41.656502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:41.659616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:41.670576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:41.695214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d056d9d5-6e98-445a-accd-9f4ec7d329bc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:41.757780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:41.758377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.758658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.758930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.761687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:41.764127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:41.769585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 201/404] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:41 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:41.774867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:41.885855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfbb6c0-f567-4b7a-83b9-f6d8e15f1627 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:41.907405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3a80ac-72ac-489e-adc4-411a678a4eee None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:41.953918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:41.954734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.955030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.955300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.975921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:41.981053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:41.981998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:41.982841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 204/405] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:41.984534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:41.985645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:41.986118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:41.986626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066/groups/ca9afdcec7a541d98c499528f44c2849/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:41.991665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] RBAC: Authorizing `identity:create_grant(project_id=15406dd00e9a441da00290b8a3dcb066, group_id=ca9afdcec7a541d98c499528f44c2849, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:41.991963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:41.994344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:42.022956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 202/406] 10.222.0.91 () {58 vars in 1442 bytes} [Wed Aug 24 03:22:41 2022] PUT /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066/groups/ca9afdcec7a541d98c499528f44c2849/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:22:42.127484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:42.131388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:42.174511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d502f9-d20d-4c53-9235-3eee717eefee None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:42.186568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:42.221931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:42.222568 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.222847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.223116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:42.223788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:42.224086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:42.225583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:42.236275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 205/407] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:42.245142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:42.378658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5fad0-20ed-4d9c-b2f2-a5adfc612424 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:42.406735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:42.414806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5f27f4-6dbf-4074-9a86-b767d3c16c7d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:42.431292 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22a495-1623-420b-94d7-209ad236729c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:42.432270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22a495-1623-420b-94d7-209ad236729c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.432756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22a495-1623-420b-94d7-209ad236729c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.433265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af22a495-1623-420b-94d7-209ad236729c None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:42.435159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22a495-1623-420b-94d7-209ad236729c None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:42.439461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af22a495-1623-420b-94d7-209ad236729c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:42.441525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 206/408] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:42 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:42.450132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22a495-1623-420b-94d7-209ad236729c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:42.500830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:42.501847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.502366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.502843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:42.527533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:42.541987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:42.543187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:42.544009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:42.546478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 203/409] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 416 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:42.554752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:42.666906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af22a495-1623-420b-94d7-209ad236729c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:42.708838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:42.709908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.710592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.711096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:42.712637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:42.714833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:42.717271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 207/410] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:42 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:42.725042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:42.749033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:42.797960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab604fec-d7ee-409d-90c7-2535ebaffe5d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:42.854645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:42.884041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:42.884660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:42.884939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:42.885239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:42.886004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:42.886359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:42.887734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:42.913792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 204/411] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:42.918554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:42.945914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7b3ba4-2080-4fd1-8388-b2e9f55ee35f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:43.020092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:43.020675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.021001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.021291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066/groups/ca9afdcec7a541d98c499528f44c2849/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.025748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] RBAC: Authorizing `identity:create_grant(project_id=15406dd00e9a441da00290b8a3dcb066, group_id=ca9afdcec7a541d98c499528f44c2849, role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:43.027224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:43.049721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 208/412] 10.222.0.91 () {58 vars in 1442 bytes} [Wed Aug 24 03:22:42 2022] PUT /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066/groups/ca9afdcec7a541d98c499528f44c2849/roles/712bb84fd46d48f28cbe60cbb255213f => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:43.152025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45776c04-3a12-413c-ac99-3985f3860644 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:43.224161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3627ff15-ed91-4f52-abfd-55efba8e3795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:43.225100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627ff15-ed91-4f52-abfd-55efba8e3795 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.225533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627ff15-ed91-4f52-abfd-55efba8e3795 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.226015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3627ff15-ed91-4f52-abfd-55efba8e3795 None admin] PATH_INFO: `/v3/groups/f0c59f00fc9a41ec8715aa498a478878` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.227993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627ff15-ed91-4f52-abfd-55efba8e3795 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c59f00fc9a41ec8715aa498a478878)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:43.234411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3627ff15-ed91-4f52-abfd-55efba8e3795 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:43.236964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 205/413] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:42 2022] GET /identity/v3/groups/f0c59f00fc9a41ec8715aa498a478878 => generated 229 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:43.243252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:43.410901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bbeef-8eda-4e68-b5fb-04bd746e18c3 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.411628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bbeef-8eda-4e68-b5fb-04bd746e18c3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.412144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3bbeef-8eda-4e68-b5fb-04bd746e18c3 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.415937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5d119-144f-4586-8184-11d89f955adf None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:43.418947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 206/414] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:43.420553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95a49-ad69-4142-ae43-6e73aef2cb81 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.421073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95a49-ad69-4142-ae43-6e73aef2cb81 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.421588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c95a49-ad69-4142-ae43-6e73aef2cb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.430147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:43.442951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:43.449938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b0c95a49-ad69-4142-ae43-6e73aef2cb81 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:43.455345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b0c95a49-ad69-4142-ae43-6e73aef2cb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:43.460733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:43.461368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.461651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.461974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] PATH_INFO: `/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.463144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] RBAC: Authorizing `identity:get_project(project_id=24a8303d4cc94c8ebcfdb44c0be112a5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:43.464633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:43.467021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 209/415] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:43 2022] GET /identity/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5 => generated 344 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:43.472799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:43.587543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 207/416] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:43.627721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ed4b90-d600-4d8f-9b89-c520e765415f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:43.666123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f67c4ae-6395-4b8c-8933-aeab42a3d1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:43.666736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c4ae-6395-4b8c-8933-aeab42a3d1be None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.667018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c4ae-6395-4b8c-8933-aeab42a3d1be None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.667697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f67c4ae-6395-4b8c-8933-aeab42a3d1be None admin] PATH_INFO: `/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5/groups/f0c59f00fc9a41ec8715aa498a478878/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.669624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67c4ae-6395-4b8c-8933-aeab42a3d1be None admin] RBAC: Authorizing `identity:create_grant(project_id=24a8303d4cc94c8ebcfdb44c0be112a5, group_id=f0c59f00fc9a41ec8715aa498a478878, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:43.671323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f67c4ae-6395-4b8c-8933-aeab42a3d1be None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:43.686488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 210/417] 10.222.0.91 () {58 vars in 1442 bytes} [Wed Aug 24 03:22:43 2022] PUT /identity/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5/groups/f0c59f00fc9a41ec8715aa498a478878/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:22:43.924587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8a8c4-dcfb-46dd-ad9e-d5d7f1d216eb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.924928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8a8c4-dcfb-46dd-ad9e-d5d7f1d216eb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.925110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8a8c4-dcfb-46dd-ad9e-d5d7f1d216eb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.926650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 208/418] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:43.933666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:43.933930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:43.934345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:43.955005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:43.956227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:44.016080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:44.020826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8a8c4-dcfb-46dd-ad9e-d5d7f1d216eb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:44.059460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 211/419] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:44.192469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8a8c4-dcfb-46dd-ad9e-d5d7f1d216eb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:44.237984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:44.243674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:44.244320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:44.244725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:44.245206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:44.258607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:44.265573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:44.266514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:44.267441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:44.268544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 209/420] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:44.275288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:44.409770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86181fdc-3aae-4d4b-a9ea-748bd82992f4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:44.440355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:44.455366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:44.456184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:44.456540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:44.456948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:44.457968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:44.458426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:44.460507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:44.482080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 212/421] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:44.488454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:44.582610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:44.677209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699dd041-1cd4-42d5-81e2-5379e1ca5bc5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:44.695136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc48e2c-694d-4875-94ab-addc0de32992 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:44.744713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:44.745247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:44.745571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:44.745899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:44.782434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ee69d-f056-4f77-9491-c7f0474b7e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:44.783763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ee69d-f056-4f77-9491-c7f0474b7e98 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:44.784248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ee69d-f056-4f77-9491-c7f0474b7e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:44.784702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ee69d-f056-4f77-9491-c7f0474b7e98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:44.797136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554ee69d-f056-4f77-9491-c7f0474b7e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-554ee69d-f056-4f77-9491-c7f0474b7e98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:44.801992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:44.803659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 210/422] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:44.805195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd77e1-b4c8-4e3a-9ce4-effe0eb0ca39 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:44.807535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd77e1-b4c8-4e3a-9ce4-effe0eb0ca39 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:44.807535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cd77e1-b4c8-4e3a-9ce4-effe0eb0ca39 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:44.807535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 211/423] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:44.813562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:44.813845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:44.814112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:44.839553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:44.841412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:44.845092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:44.847897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:44.897295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:44.901456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:44.925224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 212/424] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:44.926259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:44.971801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 213/425] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:44.983505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.043784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4dc4ed-7983-4b8f-93d0-dd6689eed60b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.102122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a68c-d658-4add-9682-85166251e694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.109544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a68c-d658-4add-9682-85166251e694 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.109544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a68c-d658-4add-9682-85166251e694 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.109544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272a68c-d658-4add-9682-85166251e694 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.109544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a68c-d658-4add-9682-85166251e694 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:45.109544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a68c-d658-4add-9682-85166251e694 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:45.109544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272a68c-d658-4add-9682-85166251e694 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:45.126591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 213/426] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:45.131446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a68c-d658-4add-9682-85166251e694 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.183473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277d1ef-7eb2-4560-afc0-5f516fd58846 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.243095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.244498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.244928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.245535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.246939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:45.248558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:45.251259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 214/427] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:44 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:45.259040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.286120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:45.294290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:45.408506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272a68c-d658-4add-9682-85166251e694 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.436029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56323aed-672c-40e2-bab0-608f27c4392c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.463866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.464573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.464936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.465429 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.488512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.489176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.489586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.490064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.491937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:45.493975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:45.496187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 215/428] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:45.497122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.578372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:45.579953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:45.598735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:45.637561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080ee574-1118-4d57-84e4-945e2b0bbcc1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.648868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:45.687130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.688562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.688995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.689452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.708302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:45.719460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:45.721476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 24 03:22:45.723222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:45.730328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 214/430] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:45.730328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.731590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 216/430] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:45.733359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.860362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fecc16-29c7-418c-9b1e-c529052092b9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.876188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12cb7e-ade1-4cda-b931-3a6ee65fa36c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:45.904220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.905164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.905634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.906098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.907409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=c50530f607054a5981b1291dff6e980b) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:45.908190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:45.910403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:45.916971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 215/431] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:45.917827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.932356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:45.939508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3858b66a-f4cb-4a01-9b6e-8d4c8635d9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:45.940766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858b66a-f4cb-4a01-9b6e-8d4c8635d9fc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.941234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858b66a-f4cb-4a01-9b6e-8d4c8635d9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.941716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3858b66a-f4cb-4a01-9b6e-8d4c8635d9fc None admin] PATH_INFO: `/v3/projects/c50530f607054a5981b1291dff6e980b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.943407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858b66a-f4cb-4a01-9b6e-8d4c8635d9fc None admin] RBAC: Authorizing `identity:get_project(project_id=c50530f607054a5981b1291dff6e980b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:45.945776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3858b66a-f4cb-4a01-9b6e-8d4c8635d9fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:45.948922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 217/432] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/projects/c50530f607054a5981b1291dff6e980b => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:45.950946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cb6a0-d310-4b3d-b472-d76318d0a8e4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:45.951440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cb6a0-d310-4b3d-b472-d76318d0a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:45.951897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cb6a0-d310-4b3d-b472-d76318d0a8e4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:45.953911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 218/433] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:45.954814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cb6a0-d310-4b3d-b472-d76318d0a8e4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:45.959783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.068317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43978c4f-0d23-4036-b209-89fc03c1e85d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.105353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cb6a0-d310-4b3d-b472-d76318d0a8e4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.128206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.129285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.129761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.130391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.131167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] RBAC: Adding query filter params (name=member) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:46.131473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:46.132838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50490ddc-aedd-4539-99e3-30e1f6d34648 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:46.143139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 216/434] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:46.144829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.145111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.145381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.154818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.155916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.156420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.156928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.158637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:46.160365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:46.163114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 219/435] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:45 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:46.163921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.169748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:46.171006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:46.259576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.263422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.269884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 217/436] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:46.270795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.304821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d98e30e-e8db-409b-bc27-90e1f9e000f8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.353198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.353833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.354108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.354397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.414276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765c2ab8-27cf-4e51-999c-8fbb92a82c84 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.439897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:46.442005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:46.459336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20704669-cec6-47b0-a525-a963690936cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.459954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20704669-cec6-47b0-a525-a963690936cd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.460236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20704669-cec6-47b0-a525-a963690936cd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.460509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20704669-cec6-47b0-a525-a963690936cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.461265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20704669-cec6-47b0-a525-a963690936cd None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=c50530f607054a5981b1291dff6e980b) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:46.461600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20704669-cec6-47b0-a525-a963690936cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:46.463109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20704669-cec6-47b0-a525-a963690936cd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:46.472624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 218/437] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=c50530f607054a5981b1291dff6e980b&role.id=d89a893a140749d39b9cecdedae58d78 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:46.474720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d12a12-be4f-450a-b861-116ed511a18c None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.475205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d12a12-be4f-450a-b861-116ed511a18c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.475664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d12a12-be4f-450a-b861-116ed511a18c None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.478140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 219/438] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:46.491404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.491841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.492445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.524786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:46.526952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:46.544966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.594440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 220/439] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:46.595700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.654013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.664907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 220/440] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:46.667079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.743181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330a324-0c49-4d3d-9c30-e3e45ea07969 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.787731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.788376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.788822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.789099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.796523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5ae7cb-2450-4e5e-af77-bcde559da339 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.803694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:46.808190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:46.809087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:46.809882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 221/441] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:46.816696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.849223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:46.850226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:46.850706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:46.851156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:46.852490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:46.854109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:46.856258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 221/442] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:46.858455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:46.862939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:46.979076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2b34e-2087-4122-a6c3-66cd5ac2cc78 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:46.983391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.040249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.041672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.041672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.041965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.042997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:47.043437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.044939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cb08b-5c1a-4497-834a-f79269bdcecd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.045096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:47.057204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 222/443] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:47.060744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.103091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.113691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.113691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.113691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.114070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.115703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:47.119834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 222/444] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:46 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:47.119834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.221995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.230042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff913c3-836d-4e81-ab6c-11ce034cbebd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.257199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ac192-c6af-4f7d-8fe2-8411b731a7e3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.267115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.267741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.268014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.268282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.279618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:47.285823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:47.287414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 223/445] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:47.288259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.294942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.303159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.303775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.304053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.304317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.391020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.393899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:47.413329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ee0c4-1a92-4a49-9aa0-e9004a84320c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.459630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.460235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.460517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.460787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.461523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=d89a893a140749d39b9cecdedae58d78, scope.project.id=15406dd00e9a441da00290b8a3dcb066) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:47.461831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.463338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:47.470233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 224/446] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=d89a893a140749d39b9cecdedae58d78 => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:47.470938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.495685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.550272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 223/447] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:47.558113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.616170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.619435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397ee548-30c2-46c7-a79b-d0ea6a7ec022 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.687547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.687850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.688153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.688392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.689339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:47.689652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.692546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:47.708279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 225/448] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:47.715119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.721703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce273298-31a9-425c-87f4-f468dbf15587 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.771011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.772331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.772331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.772331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] PATH_INFO: `/v3/projects/15406dd00e9a441da00290b8a3dcb066` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.773545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] RBAC: Authorizing `identity:get_project(project_id=15406dd00e9a441da00290b8a3dcb066)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.777220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:47.777220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 224/449] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/projects/15406dd00e9a441da00290b8a3dcb066 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:47.785903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:47.826373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.856013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186fe53-7e26-45a2-9b61-0103e6ab5c1a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.905198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.907271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.907759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.908510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.942910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:47.948942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9591-519a-4202-b7b4-9c4ba8fad594 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:47.995142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:47.995771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:47.996045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:47.996311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] PATH_INFO: `/v3/groups/ca9afdcec7a541d98c499528f44c2849` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:47.997433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] RBAC: Authorizing `identity:get_group(group_id=ca9afdcec7a541d98c499528f44c2849)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:47.998938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:48.000956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:48.001528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 225/450] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/groups/ca9afdcec7a541d98c499528f44c2849 => generated 226 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:48.002670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:48.010391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:48.113963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:48.142287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 226/451] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 24 03:22:48.149232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:48.160940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0e5f43-dc07-4d4c-bbf0-c3f3b2908229 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:48.206007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:48.207063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:48.207755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:48.208241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:48.209580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] RBAC: Adding query filter params (group.id=ca9afdcec7a541d98c499528f44c2849, role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=15406dd00e9a441da00290b8a3dcb066) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:48.210014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:48.211685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:48.219113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 226/452] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:48 2022] GET /identity/v3/role_assignments?group.id=ca9afdcec7a541d98c499528f44c2849&scope.project.id=15406dd00e9a441da00290b8a3dcb066&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 611 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:48.263232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9029b111-4f64-4f19-a209-53c59927502e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:48.302981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:48.303614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:48.303895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:48.304163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] PATH_INFO: `/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:48.305342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] RBAC: Authorizing `identity:get_project(project_id=24a8303d4cc94c8ebcfdb44c0be112a5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:48.306841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:48.308538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 227/453] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:22:48 2022] GET /identity/v3/projects/24a8303d4cc94c8ebcfdb44c0be112a5 => generated 344 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:48.315819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:48.350338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:48.436337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4583b11-b191-4101-9da4-582e27aff691 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:48.471399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63dc9f4-6050-4213-9d63-c91b10290949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:48.472029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dc9f4-6050-4213-9d63-c91b10290949 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:48.472306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dc9f4-6050-4213-9d63-c91b10290949 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:48.472576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63dc9f4-6050-4213-9d63-c91b10290949 None admin] PATH_INFO: `/v3/groups/f0c59f00fc9a41ec8715aa498a478878` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:48.473678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dc9f4-6050-4213-9d63-c91b10290949 None admin] RBAC: Authorizing `identity:get_group(group_id=f0c59f00fc9a41ec8715aa498a478878)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:48.475188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63dc9f4-6050-4213-9d63-c91b10290949 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:48.476809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 227/454] 10.222.0.91 () {56 vars in 1180 bytes} [Wed Aug 24 03:22:48 2022] GET /identity/v3/groups/f0c59f00fc9a41ec8715aa498a478878 => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:48.482820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:48.543737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:48.600273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d2312-3986-4176-a68f-649bfd02a8f3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:48.635051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:48.635646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:48.635925 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:48.636195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:48.636914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] RBAC: Adding query filter params (group.id=f0c59f00fc9a41ec8715aa498a478878, role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=24a8303d4cc94c8ebcfdb44c0be112a5) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:48.637218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:48.638693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b6297-5fb6-4986-af3e-d3525d0bfc2e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:48.644253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 228/455] 10.222.0.91 () {56 vars in 1376 bytes} [Wed Aug 24 03:22:48 2022] GET /identity/v3/role_assignments?group.id=f0c59f00fc9a41ec8715aa498a478878&scope.project.id=24a8303d4cc94c8ebcfdb44c0be112a5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:51.216309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f87e-438f-4f9f-bd9a-91c0ce694523 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:51.216309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f87e-438f-4f9f-bd9a-91c0ce694523 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:51.216309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9932f87e-438f-4f9f-bd9a-91c0ce694523 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:51.217228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 228/456] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:51.233727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68a83-d71f-4a7a-a348-4884d06d4928 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:51.235152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68a83-d71f-4a7a-a348-4884d06d4928 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:51.235569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed68a83-d71f-4a7a-a348-4884d06d4928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:51.266430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3ed68a83-d71f-4a7a-a348-4884d06d4928 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:51.267938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3ed68a83-d71f-4a7a-a348-4884d06d4928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:51.339332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0ed66-3ebd-4395-99fb-915b2b15dec4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:51.339968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0ed66-3ebd-4395-99fb-915b2b15dec4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:51.340467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0ed66-3ebd-4395-99fb-915b2b15dec4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:51.346416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 229/457] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:51.348241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690bd3-55e2-4902-a5e4-144eeb3f40ae None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:51.348737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690bd3-55e2-4902-a5e4-144eeb3f40ae None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:51.349215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29690bd3-55e2-4902-a5e4-144eeb3f40ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:51.361828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 229/458] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:51.380004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-29690bd3-55e2-4902-a5e4-144eeb3f40ae None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:51.381212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-29690bd3-55e2-4902-a5e4-144eeb3f40ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:51.484737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 230/459] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:51.642317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b20ba-8f14-4f1c-9564-d6b7c013c256 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:51.643013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b20ba-8f14-4f1c-9564-d6b7c013c256 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:51.643515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319b20ba-8f14-4f1c-9564-d6b7c013c256 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:51.645090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 230/460] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:51.653878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:51.654494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:51.654984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:51.678489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:51.680079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:51.715540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:51.772534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 231/461] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:51.836933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b20ba-8f14-4f1c-9564-d6b7c013c256 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:51.842332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:51.932290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:51.959352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bda548-23fd-4846-8e1a-a5c4c068cfb2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.023798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.024452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.024726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.024996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.044439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.045888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.052212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319b20ba-8f14-4f1c-9564-d6b7c013c256 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e8163795-7ec6-4e44-b5f4-195abe0c0ce2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:52.061089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:52.062107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:52.063588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:52.063588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.063588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:52.063588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 232/462] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:22:52.070085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060bf61-34ba-4443-a9db-066fc1e4f5ce None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.070383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060bf61-34ba-4443-a9db-066fc1e4f5ce None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.070659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060bf61-34ba-4443-a9db-066fc1e4f5ce None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.072419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 233/463] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:52.073195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060bf61-34ba-4443-a9db-066fc1e4f5ce None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.089510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:52.107338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.108485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.108839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.109722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.121729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.123367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0480a6a5-0c94-4420-a0cc-d61910f43eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:52.140657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:52.141655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:52.142537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:52.142537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.142537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:52.147050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 231/464] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:52.148828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.149118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.149512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.168954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:52.170246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:52.202590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060bf61-34ba-4443-a9db-066fc1e4f5ce None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.243194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.243915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.244203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.244521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.245164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 232/465] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:52.251013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:52.251013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.251171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.251257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.267553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 234/466] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:52.268621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.285621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:52.347335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:52.382853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6bc9b4-fcc2-421c-a0a6-b55802caf670 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.386118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8e95e-25a9-4b0f-a1cc-ec63093f228d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.423243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.423844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.424121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.424387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.425089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:52.425388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.426811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.433271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.434686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.435131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.436058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.444410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 235/467] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:52.445615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.448936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.451526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:52.473911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:52.474975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:52.475845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:52.475845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.475845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:52.475845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 233/468] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:52.476446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.559683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:52.568274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff33e72-60c0-449e-9fbc-aadb33e91109 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.586499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c862e3-4132-425c-8367-78f87cb11a59 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.606801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:52.614937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.615650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.615936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.616209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.616857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.618358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.628139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.628747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.629022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.629296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.630046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.633168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.654220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 236/469] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:52 2022] POST /identity/v3/users => generated 246 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:52.654220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.677651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 234/470] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:52 2022] POST /identity/v3/users => generated 248 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:52.679340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.869291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09af76cb-17d0-4c91-a5ec-229d4a59dc12 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.898863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05da16-f4bb-4c4c-b68e-241f67b3e811 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:52.913682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.914406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.914684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.914951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.915803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:52.916102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.917941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.931335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 237/471] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:52.937341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.938495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:52.943946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:52.944754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:52.945112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:52.945456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:52.963056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:52.964543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:52.980140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:52.981114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:52.981979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:52.981979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:52.981979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:52.982849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 235/472] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:52.989030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:52.997644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:53.057546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91ea96a-12d9-4f82-bcc1-3d0db791944a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:53.114552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb722b80-fe8e-44f0-9d82-c89e99882a85 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:53.133660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438e604-7fcf-40b4-9341-0542b1aa0592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:53.134124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e604-7fcf-40b4-9341-0542b1aa0592 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.134478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e604-7fcf-40b4-9341-0542b1aa0592 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.134704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438e604-7fcf-40b4-9341-0542b1aa0592 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.135461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e604-7fcf-40b4-9341-0542b1aa0592 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:53.136912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438e604-7fcf-40b4-9341-0542b1aa0592 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:53.168990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 238/473] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:52 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:53.170606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b318e1-12e6-4de6-93f9-f30e227f4d64 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.170894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b318e1-12e6-4de6-93f9-f30e227f4d64 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.171167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b318e1-12e6-4de6-93f9-f30e227f4d64 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.174236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 239/474] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:53.181824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.182084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.182403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.202977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:53.204051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.206717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.207332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.208280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:53.208845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:53.211528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:53.220185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:53.224492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:53.241572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 236/475] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:22:53.247953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:53.322931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:53.374576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 240/476] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:22:53.405584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33e82d7-3979-4ebb-92df-4722efedcb07 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:53.455671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:53.456283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.456557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.456822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.457477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:53.458987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:53.494764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 237/477] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:53 2022] POST /identity/v3/users => generated 249 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:53.684112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:53.727361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:53.843081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f36bc-c21e-4350-bafd-0d5dd1fac6cc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:53.887448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:53.888076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:53.888353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:53.888624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:53.889312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:53.890751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:53.900258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:22:53.900258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 03:22:53.900258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [parameters: {'id': 'c50744168ac84971afaac3be50f6a5cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 03:22:53.900258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Aug 24 03:22:53.916475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return ref.to_dict() Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:22:53.917464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:22:53.918420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:22:53.919463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:22:53.920366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:22:53.921237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'id': 'c50744168ac84971afaac3be50f6a5cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:53.924035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 24 03:22:53.926000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:53.928127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 241/478] 10.222.0.91 () {60 vars in 1131 bytes} [Wed Aug 24 03:22:53 2022] POST /identity/v3/roles => generated 144 bytes in 234 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:53.929803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:54.067553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:54.074857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77765d25-2e4d-4b60-b3ea-69515175e41f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:54.132944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7eb2dc-e438-4c53-b1a2-216508f20b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:54.133588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb2dc-e438-4c53-b1a2-216508f20b64 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:54.133878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb2dc-e438-4c53-b1a2-216508f20b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:54.134142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7eb2dc-e438-4c53-b1a2-216508f20b64 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:54.152739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7eb2dc-e438-4c53-b1a2-216508f20b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9c7eb2dc-e438-4c53-b1a2-216508f20b64 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:54.158432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:54.159296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 24 03:22:54.160123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:54.162156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 238/479] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:22:53 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:54.167302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:54.305879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:54.315873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6bf36-cb8f-417b-8025-bb014b0d5ac4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:54.362363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:54.362900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:54.363176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:54.363443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:54.364112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:54.364411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:54.365778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93fabd5-8d99-4018-b316-e19d7e827c1b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:54.374487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 242/480] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:55.029126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949afd13-1962-43fb-a1fa-af66f732ce91 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.029654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949afd13-1962-43fb-a1fa-af66f732ce91 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949afd13-1962-43fb-a1fa-af66f732ce91 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.031788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 239/481] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:55.038084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.038587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.038676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.063910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:55.065138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:55.071276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5866-5033-4a75-9d1a-f519b0611534 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.071643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5866-5033-4a75-9d1a-f519b0611534 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.071929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d5866-5033-4a75-9d1a-f519b0611534 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.073485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 240/482] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:55.079883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.080167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.080422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.123467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:55.124688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:55.177048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 243/483] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:55.215637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 241/484] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:55.527969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:55.536576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:55.702578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:55.708545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:55.717616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2997e14-0569-45b2-88ef-7c3d07e96e28 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:55.722515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e0b5e2-bb4e-42ce-82ba-11fcaa063460 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:55.756058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:55.757248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.757761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.758284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.760057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cee37-8cc5-445c-9b02-c8f6768fd2cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:55.760784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cee37-8cc5-445c-9b02-c8f6768fd2cc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.761061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cee37-8cc5-445c-9b02-c8f6768fd2cc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.761329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cee37-8cc5-445c-9b02-c8f6768fd2cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.772489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:55.773027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cee37-8cc5-445c-9b02-c8f6768fd2cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:55.777573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:55.778803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:55.779828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-236cee37-8cc5-445c-9b02-c8f6768fd2cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:55.780455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:55.781294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 244/485] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:22:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:55.787237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42869ff-2952-4ea6-9567-caa510596b2f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.788638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42869ff-2952-4ea6-9567-caa510596b2f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.788638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42869ff-2952-4ea6-9567-caa510596b2f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.788638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 245/486] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:55.788638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.789088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.789218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.790892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 242/487] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:22:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:55.791907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:55.808335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:55.809719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:55.894313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 246/488] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:55.895668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:55.907417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3b87b9-50ba-421f-9a08-b119f26063bd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:55.944440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:55.945466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:55.945995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:55.946518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:55.947423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:55.947864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:55.949406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:55.958459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 243/489] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:22:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:55.967901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:55.998053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.027666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27ccf49-18ba-410b-90be-61eac582a3ec None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.064370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.065712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.066220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.066679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.067615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:56.068165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.069939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.079511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 247/490] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:22:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:56.080204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.096574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.170213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef17ce1-a671-4e61-ae29-ca9928c5d4a7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.211754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4f7af7-6cad-4734-9501-6929ac45f238 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.221425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.222347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.222805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.223252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.234963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.236701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2d8c7742-550e-4140-b5c8-020b8ea127a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:56.259960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:56.260924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:56.261777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:56.261777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:56.261777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:56.263732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 244/491] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:56.265466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b80c8a-64b9-4a67-8949-1ac36e96d6a7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.266429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b80c8a-64b9-4a67-8949-1ac36e96d6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.266666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.267025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b80c8a-64b9-4a67-8949-1ac36e96d6a7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.267470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.267880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.268236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.269190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 245/492] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:56.270086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b80c8a-64b9-4a67-8949-1ac36e96d6a7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.285125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.286838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.294549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-445ef701-6d6d-4dd8-a530-876a015dcf9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:56.307596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:56.309379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:56.310274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:56.310274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:56.310274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:56.310274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 248/493] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:56.313375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.313774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.314150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.339078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:56.340665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:56.344691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.404241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b80c8a-64b9-4a67-8949-1ac36e96d6a7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.426839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 249/494] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:56.428143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.457224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.458444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.459241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.459720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.460992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:56.461728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.464086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.478668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 246/495] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:56.479455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.540285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a60b6-bdb5-4052-b4cb-8578ab2bc35e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.565465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.584830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.585773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.586346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.587334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.588434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:56.588919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.591073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.600129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11886-c996-419b-adbf-a568d7d09b99 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.607388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 250/496] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:56.608445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.629812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.643361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.644085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.644484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.644797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.656730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:56.662010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:56.662983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:56.663779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 247/497] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:56.664446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.742105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b6329-7792-4701-af77-a9421d6f4ea4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.778483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.779111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.779238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a118fc-d0cd-4b71-864c-65da3e792c55 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.779400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.779647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.789973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.793023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:22:56.811919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:56.813529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 251/498] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:56.814517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.823000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.823636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.823930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.824199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.831762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.833270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:22:56.846879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:56.847816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 248/499] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:56.848738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:56.866253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.902552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:56.927576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd64cd8-028a-49e7-89e1-614b4bb2574d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.963098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf4c33-38f8-48d8-86aa-8ed12c74a9be None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:56.963695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:56.964301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:56.964586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:56.964847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:56.965501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:56.965795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:56.967221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:56.975968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 252/500] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:56.979222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.002627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.003250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.003525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.003790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.013961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:22:57.019106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:57.020009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:57.020868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 249/501] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:57.021530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.081465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03fd12-b327-4006-a0c5-b254f863f866 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.083912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.119749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.120366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.120648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.120927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.121658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:57.121965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.123417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.129298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76c441-c4a7-4ab9-9ed7-45bda1f52360 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.131607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.160966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 253/502] 10.222.0.91 () {56 vars in 1137 bytes} [Wed Aug 24 03:22:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:57.161823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.169746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.170382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.170936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.170936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.171634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:57.172490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.173934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f466c-d4ac-445c-a0b3-f002bf4103de None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.209901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 250/503] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:57.211176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1580-be4f-486a-9f10-4731639361de None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.211494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1580-be4f-486a-9f10-4731639361de None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.211669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ee1580-be4f-486a-9f10-4731639361de None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.212895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 251/504] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:22:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:22:57.213583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee1580-be4f-486a-9f10-4731639361de None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.270398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1534c46b-fc20-4b3d-ba98-d1125a613f6a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.307247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.307883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.308153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.308422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.318475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.319960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.320539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ee1580-be4f-486a-9f10-4731639361de None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:57.335011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:57.335997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:57.336886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:57.336886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:57.336886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:57.336886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 254/505] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:57.337103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.353481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.359050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.359623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.359900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.360575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.360832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:57.361131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.362569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.370605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 252/506] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:57.371404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.399017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.439614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e7916-969f-44d4-8164-2d5233f61b4e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.474089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7544b24d-695e-45da-a102-c8d584eb5f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.474726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544b24d-695e-45da-a102-c8d584eb5f06 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.474826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3093a9-a1ea-4890-a233-9913c7311178 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.475007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544b24d-695e-45da-a102-c8d584eb5f06 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.475275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7544b24d-695e-45da-a102-c8d584eb5f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.508366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.508967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.509267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.509535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.542335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544b24d-695e-45da-a102-c8d584eb5f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.543859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7544b24d-695e-45da-a102-c8d584eb5f06 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.558638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.578859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.580336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.605111 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.644102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 255/507] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:57.645368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.645651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.645919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.664196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:22:57.665397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:22:57.686605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 253/508] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:57.688201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.746602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 256/509] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:22:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:22:57.747521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.803757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.812759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd73a93-3c83-4608-a26a-eb4556b3790d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.847388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:57.866548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.867675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.868148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.868607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.869800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:57.870360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.872902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.892068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b283ed-fb2a-465c-a5fc-cfbda12b9d25 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:57.900912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 254/510] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:57.900912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:57.936814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:57.937521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:57.937803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:57.938118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:57.951375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:57.953129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:57.970276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:57.971447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:57.972561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:57.972561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:57.972561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:57.976232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 257/511] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:57.977138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.020904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd145313-2a12-44ac-a612-a516cf31b04b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.060102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.060735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.061009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.061277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.074075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.075586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:58.091269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:58.092340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:58.093234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:58.093234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:58.093234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:58.096737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 255/512] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:58.098582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.109811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.116617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e551e9fe-87a6-4ab8-a66b-fd76e54cca9f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.139977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.165973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.166553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.166880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.167106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.178580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.180187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:58.195501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:58.196724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:58.197828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:58.197828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:58.197828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:58.197828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 258/513] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:58.197828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.214219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c96269-f700-4165-8bba-2e7de0c62057 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.260361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.261529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.261898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.262207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.271525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.273001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:22:58.286298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:58.287321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:58.288235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 256/514] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:58.288821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.318870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1457db-9f04-442a-b6f6-d8da69203997 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.358101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.358752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.359030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.359503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.360004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:58.360307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.361705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.374240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 259/515] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:58.375131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.387485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.400547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.408236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6480c051-805a-44a5-a668-f7b9cab3ae3a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.442951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.443497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.443773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.444042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.464006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.465498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:58.480259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:58.481299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:58.482143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:58.482143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:58.482143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:58.482143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 257/516] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:58.482404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.507494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c9729d-3421-4038-a9eb-64b309b73c90 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.546670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.548004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.548275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.548542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.549187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:58.549481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.550871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.562796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 260/517] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:58.563641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.611619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56d265e-8b27-43b9-a6c7-84a7d36a8660 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.642765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.647990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.648598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.648880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.649151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.649799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:58.650096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.651506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.664291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 258/518] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:58.665133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.676369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.685017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23341161-6fa5-4dbf-af25-7d33c6c14f25 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.724208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.724739 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.725013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.725281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.726012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:58.726336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.727776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.745969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 261/519] 10.222.0.91 () {56 vars in 1137 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:58.746553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.776275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320dd70-0377-498d-bf11-4b79e415d6ce None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.813519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.814152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.814479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.814744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.822039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.823530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:22:58.835833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:58.837254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 259/520] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/users/neutron => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:58.838165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.857224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d04a0-9f7d-4a96-a698-8acb82ddee09 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.883785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.893635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.894285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.894566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.894837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:58.895491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:58.895791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:58.897178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:58.911090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:58.916811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 262/521] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:58.917765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:58.949542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550e13d8-0c3c-4518-8a8c-b3f89c7c6374 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:58.997674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:58.998251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:58.998531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:58.998803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.005840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.007359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:22:59.019882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:59.020797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 260/522] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:59.021813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.040896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed0971e-33b7-4462-b3c0-5a0c47a6a5fb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.088656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.089644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.090052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.090552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.101625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.107337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:22:59.125449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:59.126783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 263/523] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:22:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:59.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.137761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cb3151-5659-4586-88cb-73428b4775dc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.168991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.177337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.177938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.178309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.178563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.186267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.259279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.261586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.266178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb28c13-2355-46fc-984b-3fa6c6338210 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.307204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.308228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.308696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.309143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.310325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:59.310876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.313184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.334376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 264/524] 10.222.0.91 () {56 vars in 1139 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:59.334376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.384228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 261/525] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:59.385097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.419306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.449662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1865b6-f74b-4bad-9c66-34c561be79ac None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.451030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.488561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.489171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.489448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.489735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.490382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.491837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.506121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57b7898-86eb-4a3c-a900-7bb442683c3c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.514522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 265/526] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:22:59 2022] POST /identity/v3/users => generated 247 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:22:59.515210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.567769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.568620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.568990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.569364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.570381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:59.570807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.573036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.598023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 262/527] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:59.599175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.659538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaeffa-b002-4b84-a967-caf7d6d8c37e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.699606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.700199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.700537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.700784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.701955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:22:59.702315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.704580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.710361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.728114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd8735-016e-4733-aae5-377baa234101 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.734619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 266/528] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:22:59.735709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.766014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.780158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.780778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.781059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.781926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.852339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815e74e-366c-4caa-bdcb-a45366e34591 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:22:59.856203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.857692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.891676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:22:59.892318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:22:59.892615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:22:59.892874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:22:59.903612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:22:59.905081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:22:59.921410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:22:59.922479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:22:59.923354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:22:59.923354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:22:59.923354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:22:59.923354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 267/529] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:22:59.923354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:22:59.970129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:22:59.989751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 263/530] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:22:59.989751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.003645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:00.042238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2225d-523e-4365-9a83-8798090b1809 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:00.080068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:00.080678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:00.080953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:00.081224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:00.081990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=7d0eef76ae53400c83c391351dc75c32) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:00.082391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:00.083827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:00.098476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 268/531] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/role_assignments?user.id=7d0eef76ae53400c83c391351dc75c32&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:00.098476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.147937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481719d3-d465-4dfe-a941-af00e46923ea None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:00.201417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:00.202329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:00.202751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:00.203748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:00.204649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=1d56ed96c2f546e59e473b1d62bf99eb) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:00.205132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:00.208129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:00.226498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 264/532] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:22:59 2022] GET /identity/v3/role_assignments?user.id=1d56ed96c2f546e59e473b1d62bf99eb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:00.226498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.285252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc78b79-0e0a-4fd0-b78f-4524543a77a1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:00.341804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:00.347079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:00.347712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:00.347994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:00.348370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:00.349111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:00.349418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:00.351733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:00.354726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:00.376939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 269/533] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:00.383002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.426924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb35101-ec6f-4578-9e07-9dbc509faeb0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:00.473955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:00.474838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:00.475085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:00.475440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:00.491518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:00.493480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:00.520339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:00.521358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:00.522332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:00.522332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:00.522332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:00.529162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 265/534] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:00.530941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.617593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b54c40-7416-4ab9-875f-69ec39f2f258 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:00.700164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:00.701405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:00.702094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:00.702613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:00.713334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:00.716112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:00.733722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:00.735980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:00.737543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:00.746531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 270/535] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:00.749132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.792010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6306ab2-ef18-4276-9a21-38360223964d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:00.798586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:00.874162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:00.888121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:00.888820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:00.889096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:00.889478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:00.890263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:00.890644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:00.893082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:00.928625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 266/536] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:00.935544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:00.964265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facbbcb2-d710-4bc6-a445-04a4b5a90d85 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:01.038809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:01.039483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.039764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.040025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.040752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:01.041055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:01.042633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:01.068474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 271/537] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:01.074389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:01.080808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe79663-d637-49ef-9bd2-5d4b43a4d77e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:01.119069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:01.120043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.120526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.121011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.129215 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:01.131008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:01.144623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:01.145547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:01.146466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:01.148875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 267/538] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:01.155358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:01.196544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:01.299325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498763d3-97dc-4099-80df-6970243bf38b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:01.333386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c2d3bc-2d2a-4fcd-a30a-3c5e86baf49e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:01.358732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:01.366719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:01.367332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.367609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.367875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.368574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:01.368874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:01.370354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:01.392528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 268/539] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:01.397647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:01.399553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:01.400176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.400449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.400720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.401462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=408b15d1a55a439c93a2f879c0d1cc31) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:01.401769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:01.403260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dd056e-1c7d-49eb-b38d-3018b859cf95 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:01.419562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 272/540] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:01 2022] GET /identity/v3/role_assignments?user.id=408b15d1a55a439c93a2f879c0d1cc31&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:01.499716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:01.519121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61995658-90f4-4d6a-8973-4c1bb1855480 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:01.560348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:01.561088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.561435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.561787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.562731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=5ae7930c17434ae1b6df030b0599e768) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:01.563152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:01.564981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42df0592-ec37-4a69-93aa-e45cfda7c28b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:01.574286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 269/541] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:01 2022] GET /identity/v3/role_assignments?user.id=5ae7930c17434ae1b6df030b0599e768&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:01.813852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e7a17-0299-4910-acde-eef43eb410bb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.814596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e7a17-0299-4910-acde-eef43eb410bb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.815083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70e7a17-0299-4910-acde-eef43eb410bb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.816885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 273/542] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:01.823661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:01.824209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:01.824714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:01.846937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:01.848485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:01.948845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 270/543] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:23:02.483642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e7a17-0299-4910-acde-eef43eb410bb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:02.646810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70e7a17-0299-4910-acde-eef43eb410bb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:02.695490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701d27e-c35b-4883-954f-8a9ca09ae908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:02.696393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d27e-c35b-4883-954f-8a9ca09ae908 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:02.696782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d27e-c35b-4883-954f-8a9ca09ae908 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:02.697276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701d27e-c35b-4883-954f-8a9ca09ae908 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:02.713674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701d27e-c35b-4883-954f-8a9ca09ae908 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3701d27e-c35b-4883-954f-8a9ca09ae908 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:02.719336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:02.720273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:02.721077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:02.721690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 274/544] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:02.729307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:02.854118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:02.871270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013e211-75de-45b7-b529-095dcc8e7ea7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:02.871551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013e211-75de-45b7-b529-095dcc8e7ea7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:02.871829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8013e211-75de-45b7-b529-095dcc8e7ea7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:02.874645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 275/545] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:02.879247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94cd2f-3d96-4759-b28f-43ed9dc6fc49 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:02.879533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94cd2f-3d96-4759-b28f-43ed9dc6fc49 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:02.879809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b94cd2f-3d96-4759-b28f-43ed9dc6fc49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:02.886695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a727acc-e178-4fb0-8a89-02e29b15be3d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:02.928551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8b94cd2f-3d96-4759-b28f-43ed9dc6fc49 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:02.930321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8b94cd2f-3d96-4759-b28f-43ed9dc6fc49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:02.943588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:02.944629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:02.945112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:02.945587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:02.946592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:02.946904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:02.948443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:02.958817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 271/546] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:02.964230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:02.995251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:03.040788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 276/547] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:23:03.042167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace16a10-5aae-475f-9eea-bdf17700ce07 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.042576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace16a10-5aae-475f-9eea-bdf17700ce07 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.042859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace16a10-5aae-475f-9eea-bdf17700ce07 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.044416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 277/548] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:03.060161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.060474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.060724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.099617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:03.100913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:03.105640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8a47f-aa52-48cd-b562-b5a748ee29cd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:03.144392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:03.145062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.145393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.145669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.158161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:03.160026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:03.162112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:03.176810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:03.178407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:03.179693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:03.179693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:03.179693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:03.182779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 272/549] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:03.184851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:03.265218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 278/550] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:23:03.305594 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ae656a-44a4-44ae-92c9-e7c61febd9ec None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:03.342140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:03.343185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.343469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.343818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.345017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:03.345322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:03.346883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:03.361787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 273/551] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:03.367198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:03.512624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:03.516504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd59ec5-5317-427f-aac7-06e0c0d375e8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:03.576558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:03.577640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.578708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.579223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] PATH_INFO: `/v3/users/swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.595012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:03.595294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:03.597043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:03.619868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:03.621025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:03.621964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:03.626781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 279/552] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:03.631431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:03.666848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:03.742487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ecaba-a86d-4214-a88c-53222b9871d1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:03.777478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fdb87-5b36-404c-8335-0c2ebe551fec None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:03.791091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:03.791905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.792313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.792715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.810163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:03.817647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:03.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:03.821534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:03.826448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 274/553] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:03.827344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:03.845626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:03.846276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:03.846544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:03.846811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:03.847552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:03.847849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:03.849943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:03.882914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 280/554] 10.222.0.91 () {56 vars in 1135 bytes} [Wed Aug 24 03:23:03 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:03.883621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:03.908567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:04.014691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe48ebb-feba-477b-b4a2-edd4a4d41866 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.043198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:04.055436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e4ce19-b35d-4df5-956b-22c2fc679e02 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.084224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.086221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.087790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.088513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.108179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:04.118600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:04.120159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:04.121463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:04.128873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 275/555] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:04.130429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:04.144063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.145119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.145609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.147016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.148185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:04.149191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:04.152528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d2aab-a078-41e8-98b8-aa7adabd779b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:04.168416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 281/556] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:04.170456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79828e-8e46-43a6-9a5c-9653db438c10 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.170885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79828e-8e46-43a6-9a5c-9653db438c10 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.171286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc79828e-8e46-43a6-9a5c-9653db438c10 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.172718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 282/557] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:04.173568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79828e-8e46-43a6-9a5c-9653db438c10 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:04.295956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:04.308434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed85b45-5804-4cb2-a06d-76bf0378e8e5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.334067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc79828e-8e46-43a6-9a5c-9653db438c10 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.359357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3006ad44-6cd5-442f-8c58-e15a548450d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.360430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ad44-6cd5-442f-8c58-e15a548450d9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.360930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ad44-6cd5-442f-8c58-e15a548450d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.361419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3006ad44-6cd5-442f-8c58-e15a548450d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.387073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.388122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.388612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.389087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.390035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:04.390458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:04.393102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:04.410073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 283/558] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:04.411373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:04.439309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:04.480132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006ad44-6cd5-442f-8c58-e15a548450d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:04.481596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3006ad44-6cd5-442f-8c58-e15a548450d9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:04.569923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dbb520-8621-4c19-9444-7477b8353dcc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.598213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 276/559] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:04.599511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6740-a774-42d1-b90b-d6a2fcd57470 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.599872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6740-a774-42d1-b90b-d6a2fcd57470 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.600341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4b6740-a774-42d1-b90b-d6a2fcd57470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.621484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3a4b6740-a774-42d1-b90b-d6a2fcd57470 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:04.622745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a4b6740-a774-42d1-b90b-d6a2fcd57470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:04.627438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.628339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.628794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.629181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.644616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:04.646441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:04.649099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:04.669622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:04.673393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:04.677481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:04.677481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:04.677481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:04.677481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 284/560] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:04.677481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:04.726863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 277/561] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:04.728156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0f5a-3a41-4980-9e0d-f3cc783ac92b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.728612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0f5a-3a41-4980-9e0d-f3cc783ac92b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.728750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde0f5a-3a41-4980-9e0d-f3cc783ac92b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.734251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 278/562] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:04.734251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0f5a-3a41-4980-9e0d-f3cc783ac92b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:04.741608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:04.826452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b19634-4d0f-4745-bc96-9a0c17659721 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.872071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde0f5a-3a41-4980-9e0d-f3cc783ac92b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:04.883697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.884711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.885170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.885628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.906078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:04.908346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:04.930996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:04.931734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:04.932155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:04.933863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:04.936160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:04.936160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:04.936160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:04.937131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.937131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:04.937650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 285/563] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:04.937650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:04.954639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:04.956245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b295631e-664b-4e72-b5f5-59356bd8b29c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:04.976389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:04.977520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:04.978467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:04.978467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:04.978467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:04.978467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 279/564] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:04.979884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:04.980290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:04.980723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.007187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.008667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:05.010465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:05.070887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92142c70-23d3-407f-87ac-7119b9266c39 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.072028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.091957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 280/565] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:05.093108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.117007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.118226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.118705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.119169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.120529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:05.121053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.123753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.148306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 286/566] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:05.149537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.250670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50417f6c-060a-4f5f-8e35-d0099a44d90a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.311399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.311952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979f250-87f8-4b7d-a528-5b20118bccc6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.312188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.312430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.312720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.313442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:05.315557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.317117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.321307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.337164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 281/567] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:05.338741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.367316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.368268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.368687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.369055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.369809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:05.370291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.371795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.384829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 287/568] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:05.385826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.407032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.475062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e91f9c-9650-4935-8e4a-48b0fb0f7eda None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.518954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.519905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038adf60-c48d-40ac-a821-f168366187e6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.520044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.520251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.520719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.535983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:05.541267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:05.542507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:05.543527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 282/569] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:05.549008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.566581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.567873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.568323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.568635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.581111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.584432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:05.600882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:05.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:05.603063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 288/570] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:05.603734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.635135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.676185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead33680-6d18-4690-ab4a-0a1aa1ce2183 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.680364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.718104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.718759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.719879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.720150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.728186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.729641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.730698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e124d0e4-5485-44bf-9faf-45165a77a761 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:05.741971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:05.743017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 283/571] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:05.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.772048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.772598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.772873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.773088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.787255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:05.792623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:05.793797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 289/572] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:05.794704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.865442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615f328e-d6b7-402b-8dbe-b00e897a0947 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.901465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.908171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.908850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.909342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.909617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.916556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.918034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.925700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ae7d9-853d-4913-ab46-d23e65a20caa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:05.929382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:05.931184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 284/573] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:05.932953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:05.963075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:05.971057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:05.971676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:05.971957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:05.972219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:05.972885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:05.973179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:05.974573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:05.988748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 290/574] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:05.990125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.050505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3c5bd1-11cd-49f9-b6c5-76451ee5df04 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.094742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.095370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.095658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.095931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.096693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:06.096986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.098598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.119901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 285/575] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:06.120434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe15bf4-f1ea-4a55-a3dc-de724013cfa1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.120907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.164032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.164662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.164940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.165280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.166041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:06.166371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.167825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.194083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 291/576] 10.222.0.91 () {56 vars in 1137 bytes} [Wed Aug 24 03:23:05 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:06.195326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.206673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:06.226139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:06.248529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcd5ab-1a21-4c0c-adcc-461ecec54397 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.295772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710444cd-be72-4873-8778-2a699409182d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.297147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710444cd-be72-4873-8778-2a699409182d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.297606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710444cd-be72-4873-8778-2a699409182d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.298072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710444cd-be72-4873-8778-2a699409182d None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.299648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710444cd-be72-4873-8778-2a699409182d None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:06.299648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710444cd-be72-4873-8778-2a699409182d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.300969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710444cd-be72-4873-8778-2a699409182d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.311005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 286/577] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:06.311849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710444cd-be72-4873-8778-2a699409182d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.321278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf207938-d1ce-4fba-8faf-ee16eb582d1d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.369408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.370032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.370337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.370610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.371316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:06.371616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.373058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.385910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 292/578] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:06.386780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.427689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710444cd-be72-4873-8778-2a699409182d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.466038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:06.470720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.471403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.471678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.471945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.486718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.488623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.492126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:06.506867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:06.508838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:06.510584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:06.510584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.510584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:06.510933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba994ff-935e-4256-b211-b5313e723f3f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.511060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 287/579] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:06.511060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.554838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.555964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.556206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.556660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.573403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.575056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:06.589849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:06.591433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:06.592346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:06.592346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.592346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:06.592346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 293/580] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:06.592346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.649796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c125ea1-efa1-46ef-8015-c4654b36d1fd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.689623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.690308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.690588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.690857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.702717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.704761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:06.719763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:06.720960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:06.721873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:06.721873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.721873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:06.721873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 288/581] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:06.721873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.722161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0194ae-980b-4422-8d25-ee4faddb212f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.745204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:06.768496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.769207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.769651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.769651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.783718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.785435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.788499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:06.802529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:06.803600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:06.804504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:06.804504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:06.804504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:06.804504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 294/582] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:06.804504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.859475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d06bb6b-f545-45b4-afda-fa580a7434a4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:06.896082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:06.896685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:06.896962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:06.897232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:06.897970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=c97977ebdd6f4f6da887c8a7215bda69) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:06.898306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:06.899775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:06.905588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 289/583] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/role_assignments?user.id=c97977ebdd6f4f6da887c8a7215bda69&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:06.906975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:06.964298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b94ba5-1929-41fe-8c71-ea6dcb08aaab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.015697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.016324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.016597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.016879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.017527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:07.017822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.019268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.035047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 295/584] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:07.036866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.050385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283cae0-7600-4c8e-b94c-d1de0f55e794 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.066404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.088535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.090905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.091931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.092389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.092670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.093332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:07.093632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.095617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.115277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 290/585] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:07.115277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.212278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae829144-debf-4e85-bf88-9fad13b4f354 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.247337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb57f9d6-ad07-4ff0-b1f6-ad0ea67db6f9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.264113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.264732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.265012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.265286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.266502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:07.266805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.268175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.280873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 296/586] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:07.282343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.299940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.300587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.300879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.301133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.301783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:07.302119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.303980 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.322224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 291/587] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:07.322224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.368956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.407087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.430011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6525f85f-1996-4acc-94c2-eed2568b16db None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.476356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.476992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.477271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.477544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.479331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2224d9fb-77bd-4632-9859-0d56c3121d85 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.487581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.489053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:07.503743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:07.505710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 297/588] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:07.506723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.541125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.541809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.542092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.542535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.550939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.552419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:07.565536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:07.567456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 292/589] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:07.568989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.664556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2234e5d-d4ed-4bc7-bcea-495b656e921b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.686796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.705682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ba546-87ae-4bd5-9105-a1e9bbd6c5da None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.706692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.707339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.707683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.707871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.721051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.722677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:07.736117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:07.737791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 298/590] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:07.739451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.756568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:07.759048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.759670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.759946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.760215 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.769758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.771271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:07.784593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:07.785733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:07.786739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 293/591] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/users/neutron => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:07.787880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.864345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeebc735-ba6c-4c63-8cf1-38ed76ba7f71 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.905524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.906130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.907050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.907459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.908064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:07.908367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.910075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.921437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ea0906-b190-4de0-8ec7-ad4d75622252 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:07.928400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 299/592] 10.222.0.91 () {56 vars in 1137 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:07.929264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:07.977071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:07.978266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:07.978678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:07.979338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:07.980619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:07.980935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:07.982480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:07.998855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 294/593] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:08.005666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.030264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:08.032100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:08.076051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a98634-954c-4a5a-8501-dd53a44d6544 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.132103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.133008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.133354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.133682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.136067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:08.136067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.137181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.155430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 300/594] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:08.155430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.176280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f319a420-4f1d-4615-9778-a8126146a727 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.221165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.221876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.222159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.222455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.223187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:08.223490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.225020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.252789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 295/595] 10.222.0.91 () {56 vars in 1139 bytes} [Wed Aug 24 03:23:07 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:08.252789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.308617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46bbbe3-bb20-489b-b9f9-d398aeb16c0d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.337808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:08.342313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:08.352598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.353449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.353619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.354023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.366477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.367962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.376492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855fef3d-0cf9-4992-8687-72afed2cbf1a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:08.383236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:08.384556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:08.385872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:08.385872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:08.385872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:08.389271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 301/596] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:08.390001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.424647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.425298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.425571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.425837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/1d56ed96c2f546e59e473b1d62bf99eb/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.441394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=1d56ed96c2f546e59e473b1d62bf99eb, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.442958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.460101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 296/597] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:08 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/1d56ed96c2f546e59e473b1d62bf99eb/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:08.461010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.523176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ecdaa-225a-43a9-8fb5-9c4fdf2f8388 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.580745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.582029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.582533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.582823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/7d0eef76ae53400c83c391351dc75c32/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.608181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=7d0eef76ae53400c83c391351dc75c32, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.609747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.627826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 302/598] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:08 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/7d0eef76ae53400c83c391351dc75c32/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:08.628978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.636363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f14768-fffa-4cb4-8b0b-186db439f2c6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.669709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:08.686626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:08.692825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.694301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.694737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.695262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.718383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.722884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:08.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:08.755011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:08.756123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:08.756123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:08.756123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:08.756123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 297/599] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:08.763747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.819459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe83d5-9e1a-4c9e-aa32-51089feacf3a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.870574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.872078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.872078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.872296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.872986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:08.873289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.875467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.892903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 303/600] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:08.898964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:08.928342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665c513c-bc6b-4c4b-9721-7490f3b9f19b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:08.976446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:08.976959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:08.977409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:08.977688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:08.978745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:08.979060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:08.980979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:08.996643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 298/601] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:09.003043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:09.030001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f701bef1-d5ef-4161-8f45-6bc1506de77d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:09.038568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:09.046924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:09.083110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:09.084588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.084880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.085432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.095482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:09.096968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:09.110800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:09.112018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:09.113198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 304/602] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:09.121932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:09.159554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4266cae9-cdec-45eb-b8f8-78b958869d69 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:09.212163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:09.212879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.213218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.213374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.221884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:09.223538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c4f284a5-c430-4664-b7ed-549ea9439f6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:09.237357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:09.238493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:09.239619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 299/603] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:09.240506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38e842-30a7-4d9a-bef3-a0e08abd11aa None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.240506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38e842-30a7-4d9a-bef3-a0e08abd11aa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.240718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38e842-30a7-4d9a-bef3-a0e08abd11aa None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.241891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 300/604] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:09.246767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38e842-30a7-4d9a-bef3-a0e08abd11aa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:09.296614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c8c0d1-09da-4b99-8ab8-cdae2cbdb7cc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:09.343321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:09.343991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.344785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.345060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.345761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:09.346097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:09.347589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce50cfe7-6e67-4753-93e6-0aa905e4a580 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:09.364593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 305/605] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:09.366278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.366954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.367460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.370881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:09.381009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38e842-30a7-4d9a-bef3-a0e08abd11aa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:09.390706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:09.392836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:09.396539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:09.425583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:09.426529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.426865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.427145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.427907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:09.428300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:09.430298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:09.450277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 301/606] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:09.450277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:09.479219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 306/607] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:09.480483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:09.665132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469e39c8-f018-46f8-9db5-7ade67157585 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:09.667674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9b016a-c211-4f19-b80d-433337dbdd1c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:09.733240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:09.734630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.734800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.735086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/5ae7930c17434ae1b6df030b0599e768/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.738517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:09.739173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:09.739449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:09.739719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/408b15d1a55a439c93a2f879c0d1cc31/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:09.759679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:09.769141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=5ae7930c17434ae1b6df030b0599e768, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:09.770990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:09.775239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=408b15d1a55a439c93a2f879c0d1cc31, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:09.776921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:09.797774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 307/608] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:09 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/5ae7930c17434ae1b6df030b0599e768/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:09.806696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 302/609] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:09 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/408b15d1a55a439c93a2f879c0d1cc31/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:09.831839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:09.993573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cf411e-93b5-48af-8dbc-74a3b433fbc9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:10.055839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:10.056901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:10.057335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:10.057743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:10.072403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:10.081819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:10.082887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:10.083747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:10.084423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 308/610] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:10.095358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:10.095697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:10.216283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdcacce-e04e-4143-a96a-87f379962523 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:10.263205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:10.263720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:10.263991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:10.264258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:10.264926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:10.265224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:10.266684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:10.275809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 303/611] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:10.281300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:10.347517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:10.416249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a841d-e2e0-4a83-94b9-5b09397e3dc9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:10.470394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:10.471041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:10.471320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:10.471609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:10.483083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:10.484598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-38286310-9a33-4864-a32a-45b0a4720841 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:10.505607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:10.506833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:10.507733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:10.507733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:10.507733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:10.507733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 309/612] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:10.516044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:10.604977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:10.661334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77812a-980b-4585-aa75-16024294d8be None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:10.718775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cef06-fc43-4a1e-8274-869565c05589 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:10.719139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cef06-fc43-4a1e-8274-869565c05589 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:10.719352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101cef06-fc43-4a1e-8274-869565c05589 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:10.720630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:10.720965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 310/613] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:10.721768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:10.722310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:10.722808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:10.723681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:10.724187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:10.725859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:10.726598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:10.726882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:10.727154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:10.758158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 304/614] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:10.759490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:10.760778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:10.763716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:10.860276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 311/615] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 24 03:23:10.985823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bdf1a-e9a5-4bfa-a222-81dcb773f6ee None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.038544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:11.067053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.068148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.068646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.069142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.080009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:11.081476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:11.098143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:11.100013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:11.101921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:11.111557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 305/616] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:11.111788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.159255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.160073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:11.250987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad66642-cdde-4e1d-9ebc-ce8a5f638f13 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.294413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.295493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.295769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.296038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.296916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:11.297260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:11.298741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:11.325697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 312/617] 10.222.0.91 () {56 vars in 1135 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:11.327276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.343398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347956ea-921e-478f-b25c-77e5d931fec5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.379472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416f4f89-4595-4ca0-941a-81f3489c934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.380009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f4f89-4595-4ca0-941a-81f3489c934a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.380288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f4f89-4595-4ca0-941a-81f3489c934a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.380898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416f4f89-4595-4ca0-941a-81f3489c934a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.393259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416f4f89-4595-4ca0-941a-81f3489c934a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-416f4f89-4595-4ca0-941a-81f3489c934a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:11.405624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:11.406727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:11.407569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:11.413366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 306/618] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:11.415611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d1d81-4eda-4121-9aa8-27fd3ba08ef7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.416075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d1d81-4eda-4121-9aa8-27fd3ba08ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.416529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956d1d81-4eda-4121-9aa8-27fd3ba08ef7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.421676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 307/619] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:11.423078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d1d81-4eda-4121-9aa8-27fd3ba08ef7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.446537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:11.461479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5d5a48-461e-4244-843f-ce55dde886d8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.494578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:11.514506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.515537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.516541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.517181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.529607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:11.531550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-50b92dd8-0a32-4a4d-b469-9380d1a95852 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:11.549193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:11.550675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:11.551823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:11.551823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:11.551823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:11.554364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 313/620] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:11.554857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.555278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.555637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.572526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956d1d81-4eda-4121-9aa8-27fd3ba08ef7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.579937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:11.581255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:11.622165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.624084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.624579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.625051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.626494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:11.627000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:11.629920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:11.642903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 308/621] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:11.644825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.671865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 314/622] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:11.682870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.775221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2033c7-3dca-494c-9a23-b2c1cc0590ad None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.784217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:11.825246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.826042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.826449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.827869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:11.829200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:11.830122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:11.830699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:11.832386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:11.848107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 309/623] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:11.854451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:11.905584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101e3bb-eea6-4b14-b7d2-85990e02e221 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:11.988052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:11.988703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:11.988979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:11.989250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.010993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.012622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:12.042354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:12.044216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:12.046995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:12.046995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:12.046995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:12.046995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 315/624] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:12.054421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.064405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b558374-1e2b-4b23-a6f6-5fc52aff2334 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.135502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.136144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.136422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.136844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.147704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.150810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:12.166929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:12.167930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 310/625] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:12.169262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.204561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.236517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f062363-4917-4a0d-ac00-35f27366081a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.283923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.299738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.301148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.301925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.302466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.303607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:12.304138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.306445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.308394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7a5480-e599-464b-b6fa-dbb297aedd7e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.322958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 316/626] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:12.324231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.356737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.358083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.358642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.359119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.370763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:12.376786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:12.377783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:12.378687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:12.381066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 311/627] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:12.382202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.465494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325bbcb7-c927-4849-b3f2-47d3211612aa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.481309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.507451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076aab42-2929-4bae-a48e-ca28e69608e2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.521744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.522401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.522708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.522949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.523648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:12.523948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.525375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.538354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 317/628] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:12.539205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.553492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.554127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.554447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.554718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.562527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.564550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-dc35c902-39fa-4e80-b602-5c8d3e111d8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:12.576086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:12.577095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:12.578084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 312/629] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:12.578902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feaab6-4d8b-4d42-8e94-6035ce49d904 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.578902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feaab6-4d8b-4d42-8e94-6035ce49d904 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.578902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38feaab6-4d8b-4d42-8e94-6035ce49d904 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 313/630] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:12.583053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feaab6-4d8b-4d42-8e94-6035ce49d904 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.599323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.661744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ab2bfb-2b47-4ea6-b10a-18e24f00e2f8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.696420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38feaab6-4d8b-4d42-8e94-6035ce49d904 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.701430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.702036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.702350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.702625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.703293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:12.703622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.705347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50dcf654-63c4-4720-8cda-e9565817c5b6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.714505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 318/631] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:12.716308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152f152-03f1-4e58-9791-0010ebbdea37 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.716546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152f152-03f1-4e58-9791-0010ebbdea37 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.716881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152f152-03f1-4e58-9791-0010ebbdea37 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.720916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 319/632] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:12.720916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152f152-03f1-4e58-9791-0010ebbdea37 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.737042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6b060-6c0f-427d-9e79-bfacf1ac3cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.737697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6b060-6c0f-427d-9e79-bfacf1ac3cf0 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.737978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6b060-6c0f-427d-9e79-bfacf1ac3cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.738273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6b060-6c0f-427d-9e79-bfacf1ac3cf0 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/c97977ebdd6f4f6da887c8a7215bda69/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.753847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6b060-6c0f-427d-9e79-bfacf1ac3cf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=c97977ebdd6f4f6da887c8a7215bda69, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.755733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd6b060-6c0f-427d-9e79-bfacf1ac3cf0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.760535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.773443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 314/633] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:12 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/c97977ebdd6f4f6da887c8a7215bda69/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:12.775220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.775664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.776078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.808434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:12.810705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:12.852903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152f152-03f1-4e58-9791-0010ebbdea37 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:12.864077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:12.910216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 315/634] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:12.910216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:12.915169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:12.915998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.916342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.916833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.917855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:12.918276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:12.919929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a882f89d-9931-4835-957e-3c9238fbdcc0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:12.943668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 320/635] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:12.945454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:12.945963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:12.946497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:12.966545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:12.967678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:13.052853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a426d1-ccdf-47a7-b784-2ec3c2af6d01 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.069291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 321/636] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:13.070242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.085251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:13.097969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.099245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.099989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.100798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.119856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:13.127038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:13.144948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:13.145978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:13.146902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:13.146902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:13.146902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:13.153730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 316/637] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:13.155421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.182645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:13.188619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0414a6ed-87e3-43bc-bd90-e306f004da04 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.232363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.232894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.233170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.233435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.291841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a8c194-7091-4e79-80a6-0ea7bf68cfe8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.329017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.330071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.330640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.331159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.332151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:13.332669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:13.334358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:13.352682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:13.353369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 317/638] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:13.353635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:13.354782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.401615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:13.472455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:13.485400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c3e117-ef8c-44e9-93dc-9e494357d440 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.492257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 322/639] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:13.492257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.539440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.540099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.540489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.540799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.552272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:13.559485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:13.561724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 318/640] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:13.564396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.626930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2aade6-86e5-4706-9da1-e2a1d26d3fc7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.676902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.677556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.677826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.678104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.682308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:13.686926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69940c8f-09c2-4021-b964-271f744bd84e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.687078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:13.689094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:13.704414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:13.705553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 323/641] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/users/glance => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:13.706800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.754510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.755089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.755436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.755776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.770442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:13.777389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:13.778557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 319/642] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:13.783495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.805299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:13.865294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc92826-3823-4bfb-b8a5-8b095d3d02a7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.925416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d466b6-fe56-41a8-981d-24c959ff24c5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:13.928467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.929103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.929374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.929640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.943874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:13.945331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:13.961153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:13.963270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:13.964517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:13.964517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:13.964517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:13.966904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 324/643] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:13.972648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:13.978626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:13.979339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:13.979697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:13.980041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:13.980801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:13.981178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:13.982675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:13.996535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 320/644] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:13.997444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.024422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:14.120708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a003b-2ba7-44e3-8180-9c32e942f6c3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.129521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:14.134704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e153e7-e62e-4069-9c8f-b2f00d86f007 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.168577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.169689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.170152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.170699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.173493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:14.174031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.176768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.185252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.186860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.187337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.187895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.189342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:14.189872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.191843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.201473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 321/645] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:14.202683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.212354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 325/646] 10.222.0.91 () {56 vars in 1137 bytes} [Wed Aug 24 03:23:13 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:14.213629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.320977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:14.341085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77035e06-f270-428c-88b7-8caa836b6050 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.361902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668a9cb9-cbd6-4fad-9073-044df07928f6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.383035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.383748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.384031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.384306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.384955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:14.385251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.386777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.402649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 322/647] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:14.403883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.414496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.415223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.415640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.415911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.438233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.440141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:14.458558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:14.460111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:14.461773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:14.461773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:14.461773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:14.461773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 326/648] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:14.461773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.491081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:14.555505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54b07ab-5392-4310-9315-b0b06711a885 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.607723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd151fbc-d0e5-4c57-be57-a31e0371c8f0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.628457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.629615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.630067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.630492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.646141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.648914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.651997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:14.653469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.654695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.655155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.655665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:14.677696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:14.685461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:14.686644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:14.686644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:14.686644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:14.686644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 323/649] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:14.686644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.758140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.760744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.795698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:14.838475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e013a-25f9-4815-a8e6-d89bbef9f7a0 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:14.880215 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:14.880762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:14.881064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:14.881332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:14.888847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:14.890357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:14.908294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:14.910536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 324/650] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:14.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.936937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 327/651] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:14.937936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:14.994254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.056327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71638da9-908a-46a3-a31a-55de45fb049c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.074002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fab06e5-b765-46a0-845f-b37d16490d45 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.104175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.110632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.111342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.111696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.111965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.112774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:15.113621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.114593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065ce96-ea61-4205-b8a8-83706e12f396 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.128305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.129135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 325/652] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:15.129317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.129755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.130154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.131343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:15.131456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3243e-f00a-419d-be7d-00ba597af4aa None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.131790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3243e-f00a-419d-be7d-00ba597af4aa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.131895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.132275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef3243e-f00a-419d-be7d-00ba597af4aa None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.134515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.134654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 326/653] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:15.135945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3243e-f00a-419d-be7d-00ba597af4aa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.153683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 328/654] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:15.154956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.262570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef3243e-f00a-419d-be7d-00ba597af4aa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.277631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.301185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e117cb09-169e-4c3e-a6a7-6ab512418fff None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.304331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.305282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.305747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.306219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.308830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:15.309362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.312072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.338828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 327/655] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:15.338828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.348369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.349414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.349886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.350394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.368951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.371621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8a04b35b-1127-4b33-92e5-9ed9ff1cada9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:15.396566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:15.397566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:15.398496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:15.398496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:15.398496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:15.398496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 329/656] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:15.400449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.400915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.401372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.430199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:15.432214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:15.444586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af369fd4-f96d-4a29-8b51-3ef9f3f95b19 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.471435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.480976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.481605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.481883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.482153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.490476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.492002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:15.503750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:15.504867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 328/657] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:15.506154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.542540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.544059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 330/658] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:15.544907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.617953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499a868-8e89-4d38-8cd5-5eda4178c0b9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.655967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.656507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.657343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.657615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.666351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.666933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a8fdf-6c2a-42cd-a6d4-08b88e469105 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.667915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:15.682437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:15.683482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 329/659] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:15.684438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.709623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.710275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.710631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.710811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.711537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=1d56ed96c2f546e59e473b1d62bf99eb) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:15.711842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.713283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.719176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 331/660] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/role_assignments?user.id=1d56ed96c2f546e59e473b1d62bf99eb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:15.719951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.748896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.818633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:15.822946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c45d8-c0bb-42eb-a111-f3432ebcad7c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.860657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943822e6-f053-4f6b-b751-b77f625c8505 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:15.886672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.887652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.888250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.888589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.889737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:15.890302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.893250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.909987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 330/661] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:15.910943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:15.912737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:15.913663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:15.914112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:15.914587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:15.915836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:15.916336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:15.918867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:15.940434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 332/662] 10.222.0.91 () {56 vars in 1137 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:15.943924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.048005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.054934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd82d7-a22a-422c-a640-598fb16b8cef None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.068714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09098dec-0974-4038-86ac-50dc9f2ae33f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.107121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:16.107121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:16.107121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:16.107121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:16.107548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:16.107548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:16.108584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:16.112733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:16.113500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:16.113790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:16.114079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:16.130451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:16.135764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:16.136992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:16.137865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 333/663] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:16.138623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 331/664] 10.222.0.91 () {56 vars in 1139 bytes} [Wed Aug 24 03:23:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:16.138700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.138777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.155200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.288719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c67af3-9e61-4f57-92bd-0d69d6bbee52 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.289081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd082d-af57-48ad-bebb-b2df493cc596 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.344417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:16.345197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:16.345342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:16.345657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:16.352916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:16.354040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:16.354576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:16.355059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:16.363237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:16.364674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:16.368054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:16.383122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:16.384107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:16.385062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:16.388267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 334/665] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:16.389099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.443939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:16.445451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:16.459806 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.552416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65d9246-d5f0-446e-ba29-8bb07f253ee8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.564368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 332/666] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:16.564368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.605473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:16.606103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:16.606490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:16.606929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:16.694919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.702860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:16.705887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:16.738846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0d29-d0ef-4ee2-9e42-a4ad2cd3be43 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.755374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.784047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:16.784973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:16.785246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:16.785558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:16.786258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:16.786561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:16.788097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:16.802347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 333/667] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:16.803246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.851644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 335/668] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:16.852766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:16.958926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb8a77-5b14-423d-8bf8-438f497c2df1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:16.981933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:16.987585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad11ad8-f238-447e-85cb-d36792ea4436 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.004402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.005013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.005297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.005545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.006343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:17.006639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.008094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.021425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 334/669] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:17.022516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.037617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.038483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.038949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.039382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.053036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.055573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.073119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:17.076269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:17.078052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:17.080783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:17.080783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:17.080783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:17.080783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 336/670] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:17.081257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.191193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24824c96-63fa-4594-bfc0-2559145f11f7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.235470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd2b6-afec-4503-a32b-d0dd4540d96b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.242037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.242872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.243152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.243421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.255945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.257553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:17.272855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:17.274905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:17.276189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:17.276189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:17.276189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:17.276189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 335/671] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:17.276189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.295859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.296978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.297259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.298096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.313904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.316639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:17.344925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:17.346670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:17.347922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:17.347922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:17.347922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:17.347922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 337/672] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:17.348779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.356617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.423620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.441521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd2e183-f115-4408-ab10-ee4f279129a1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.504023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.505011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e2f38a-88da-494f-8097-7509a1af7f0a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.505199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.505530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.506000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.507406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=7d0eef76ae53400c83c391351dc75c32) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:17.507955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.510749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.521901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 336/673] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/role_assignments?user.id=7d0eef76ae53400c83c391351dc75c32&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:17.523078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.568599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.570003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.570788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.571538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.572889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:17.573753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.576683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.601214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 338/674] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:17.603201 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.721364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.740300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d053f3cb-6ae2-44e3-a0ec-968a94aaf721 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.789220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec547c02-d295-46df-9fe1-801685912b5f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:17.807289 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.808164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.808455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.809243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.811044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:17.811575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.813890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.829130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 337/675] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:17.834009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.844011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:17.849758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:17.850607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:17.850956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:17.851378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:17.852355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:17.852543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:17.856240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:17.875011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 339/676] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:17.877099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:17.961612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9ada7-9b2c-4ec1-9613-bc9e1d0a0864 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.004112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.005127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.005624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.006099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.013441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.015150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:18.027319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:18.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:18.029140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:18.030033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889680a7-e0e4-40dc-a27d-3c8f8f84553f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.037928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 338/677] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:18.037928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.083061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.114067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.114740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.115020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.115303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.124558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.126960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.144574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:18.147658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:18.154108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 340/678] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:18.156256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.196531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45335e9-3dcc-4538-8a72-4b8e0bbf7ca6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.245147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.246156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.246667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.247143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.259816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.261294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-929bc074-f391-44c5-a50b-2861c3bd644a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:18.273554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:18.274880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:18.275920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:18.279545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 339/679] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:18.280774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e55d3-84e3-41ff-8abc-80d80625e993 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.281048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e55d3-84e3-41ff-8abc-80d80625e993 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.281318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e55d3-84e3-41ff-8abc-80d80625e993 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.283473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 340/680] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:18.283938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e55d3-84e3-41ff-8abc-80d80625e993 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.303062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd1d8e1-b2ca-40cd-9fd8-f2e04a848d63 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.345426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.346648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.347378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.347663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.348499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:18.348799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.350261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.364138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 341/681] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:18.369891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.412120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e55d3-84e3-41ff-8abc-80d80625e993 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.432421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.443909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.450552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.451136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.451776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.451776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.452418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:18.452715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.454733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e7ee5-411d-4991-a3f0-cf8e41b02981 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.482581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 341/682] 10.222.0.91 () {56 vars in 1135 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:18.484865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.485442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.486003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.510660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:18.511998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:18.529529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fdbbdf-8d2c-416d-a885-8d2a539c28b7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.582583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.583854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.584512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.585238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.586784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:18.587305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.590002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.610214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 342/683] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:18.611014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.628794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 342/684] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:18.628794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.755353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e83078-1b0c-4322-b32a-25bf5edf9089 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.781094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d19f2a-0b2b-4fd2-bf61-c7178c33826a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:18.781799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.805477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.806851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:18.808318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.809289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.809965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:18.811215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=408b15d1a55a439c93a2f879c0d1cc31) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:18.811757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:18.814859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:18.825398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 343/685] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/role_assignments?user.id=408b15d1a55a439c93a2f879c0d1cc31&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:18.827160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:18.849861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:18.850526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:18.850807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:18.851078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:19.033725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352a1ed-bb37-48d1-9e3a-b3aa0e5649f6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:19.067192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:19.069736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:19.098843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:19.100285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:19.100619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:19.101101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:19.103463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=5ae7930c17434ae1b6df030b0599e768) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:19.103463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:19.107132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:19.122307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 344/686] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/role_assignments?user.id=5ae7930c17434ae1b6df030b0599e768&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:19.123802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:19.183343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:19.205285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 343/687] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:19.213458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:19.259708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:19.286549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28bb41d-1cea-4abd-94df-70b116b6b26b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:19.328921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:19.329567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:19.329816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:19.330064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:19.346209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:19.348947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:19.350654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:19.352444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 345/688] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:19.363821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:19.516272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:19.525056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a465b292-b88d-44a4-ac26-b47e7052c9d9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:19.526850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7467325c-fb10-41f1-9fc4-f98a5109d6d3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:19.571681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:19.572431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:19.572798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:19.573175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:19.574138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:19.574570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:19.576822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:19.592716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 346/689] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:19.599625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:19.617554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:19.619041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:19.619233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:19.619618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:19.645231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:19.648490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:19.684921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:19.686813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:19.689483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:19.689483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:19.689483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:19.689483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 344/690] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 477 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:19.697622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:19.907369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509224a-3b1c-40ce-88b8-a29011a03452 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:19.912033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:19.968808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2069fecb-88c7-40a8-a992-66b52fa9f438 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:20.015820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:20.017827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.019132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.020129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.020995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:20.022230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:20.023456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.023975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:20.025593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:20.026089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.026726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.049976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:20.052228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:20.057375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 345/691] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:20.064526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:20.085163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:20.086358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:20.087280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:20.087280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:20.087280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:20.099280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 347/692] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 501 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:20.109300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:20.137007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:20.261323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f174b-dc23-48ff-8e23-af16dde65925 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:20.319306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:20.343986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:20.346120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.349228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.349713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.363455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:20.366011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-aa856290-eea6-4083-b255-61818e320ec5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:20.386941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:20.387901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:20.389220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:20.397414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 346/693] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:20.399601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd572be-82d8-4345-a196-7e6c644b32b6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.399966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7390648e-1de6-486d-a184-4606c7692341 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:20.400182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd572be-82d8-4345-a196-7e6c644b32b6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.400614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd572be-82d8-4345-a196-7e6c644b32b6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.403307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 347/694] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:20.404580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd572be-82d8-4345-a196-7e6c644b32b6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:20.469471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:20.470590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.471163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.471727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.472803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:20.473579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:20.476962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3f6eaa-3005-40bd-9d29-7bf1ab15ddcf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:20.501793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 348/695] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:20.503841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.504298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.504744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.539908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:20.541922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:20.587338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd572be-82d8-4345-a196-7e6c644b32b6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:20.648743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:20.649690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.650317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.650870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.652066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:20.652701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:20.655403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:20.662864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:20.682216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 349/696] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:20.682216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:20.689345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 348/697] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:20.699171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:20.858671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:20.881251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24db0da-3b12-465c-9406-50c501bda25c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:20.947213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:20.948854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:20.949579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:20.950453 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:20.965306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:20.968469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:20.992131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:20.993963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:20.995831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:21.000475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 350/698] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:23:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:21.009340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:21.025315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6510ec-952f-483e-9963-860a2f8edbd7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:21.178919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:21.179546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.179836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.180109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.180855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=c97977ebdd6f4f6da887c8a7215bda69) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:21.181167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:21.182656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:21.200368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 349/699] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:20 2022] GET /identity/v3/role_assignments?user.id=c97977ebdd6f4f6da887c8a7215bda69&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 502 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:21.207634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:21.248588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b78633a-11cd-4238-8f99-2e7c58f1156a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:21.311478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:21.339541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:21.340924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.341658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.342586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.344760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:21.345710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:21.348407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:21.404260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 351/700] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:23:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 396 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:21.415881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:21.542909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c48d462-5fbf-4f05-a537-afa6964997a9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:21.573987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:21.610724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc40491-e1c4-404c-9e95-51607fbe5adf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:21.621317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:21.622325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.622739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.623153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.639002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:21.643830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:21.647125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0e8b183e-ce1e-4c38-8d8e-fc6b1d3fed51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:21.675618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:21.676787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:21.677893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:21.677893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:21.677893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:21.677893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 350/701] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:21.679752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d98114-44ad-47b8-976e-86656c74840a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.680503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d98114-44ad-47b8-976e-86656c74840a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.680503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d98114-44ad-47b8-976e-86656c74840a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.682688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 351/702] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:21.683744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:21.684836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.685317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.685786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.688041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d98114-44ad-47b8-976e-86656c74840a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:21.810998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:21.813758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:21.828024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d98114-44ad-47b8-976e-86656c74840a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:21.886441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:21.887348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.887802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.888313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.889418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:21.890087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:21.892921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d9806-7311-4914-aa13-f4b2b7d39621 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:21.922008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 352/703] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:21.922008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:21.922008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:21.922008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:21.952827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 352/704] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:21.953911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:21.954378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:21.965558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:21.973392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:22.008995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:22.101025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303c9c78-9b84-4bf2-bc68-f21c6eb0eded None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:22.102057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 353/705] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:22.105722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:22.154626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a559c-511e-425d-ad90-5bafc62b2b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:22.155590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a559c-511e-425d-ad90-5bafc62b2b8c None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.156025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a559c-511e-425d-ad90-5bafc62b2b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.156481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a559c-511e-425d-ad90-5bafc62b2b8c None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.158297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a559c-511e-425d-ad90-5bafc62b2b8c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:22.160933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a559c-511e-425d-ad90-5bafc62b2b8c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:22.201413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 353/706] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:23:21 2022] POST /identity/v3/users => generated 254 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:22.201413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09856212-4cd4-4b5e-93a3-1f60a8c12cd7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.201413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09856212-4cd4-4b5e-93a3-1f60a8c12cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.203062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09856212-4cd4-4b5e-93a3-1f60a8c12cd7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.206878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 354/707] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:22.223682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.224312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.224312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.267720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d06b0-1583-4348-96b7-9fe6e62a93ae None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:22.268167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:22.270074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:22.318756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:22.321127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.321673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.322206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.338056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:22.340299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:22.359034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:22.360134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:22.361072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:22.361072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:22.361072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:22.363357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 354/708] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:22.376048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:22.399031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:22.468520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:22.478648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 355/709] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:22.482435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:22.560041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1154d7c2-4030-4090-9149-303bf13be9c1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:22.610489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21d6aa9-0a59-46c4-9bc8-f1a80ec90552 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:22.637275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:22.638013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.638330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.638474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.639231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:22.639538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:22.640938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:22.661415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:22.662533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.663022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.663497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.664321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:22.665893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:22.666006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 355/710] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:22.676853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:22.683717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:22.684834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:22.685723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:22.689340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 356/711] 10.222.0.91 () {56 vars in 1129 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:22.696005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:22.793871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:22.821163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:22.849652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba1c2-2f96-4985-97ce-db51a9e15d87 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:22.881762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937215a7-7575-4279-9562-82d2be7f2090 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:22.898863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:22.899763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.900212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.900556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.901308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:22.901680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:22.903291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:22.913783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 357/712] 10.222.0.91 () {56 vars in 1124 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:22.914792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:22.934679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:22.935602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:22.935884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:22.936155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:22.948352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:22.949910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:22.966390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:22.969550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 356/713] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:22.973089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:23.075802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660fb82-8110-47ca-ad87-cd436852b556 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:23.114370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:23.131302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:23.132716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.133698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.134296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] PATH_INFO: `/v3/services/network` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.135697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:23.137187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d97efa3-0e67-48bc-98e5-3c72d1ba3429 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:23.138297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:23.151244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:23.153589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:23.155719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 358/714] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:23.157652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:23.189149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:23.190177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.190821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.191387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.192236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:23.192746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:23.195630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:23.196868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:23.214360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 357/715] 10.222.0.91 () {56 vars in 1124 bytes} [Wed Aug 24 03:23:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:23.216249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:23.308675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c75f4af-1ba0-41f4-841c-5120961a0c06 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:23.369673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:23.372715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.373199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.373480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.374207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:23.374516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:23.376138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:23.412208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 359/716] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:23.421935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:23.439869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c357a83e-453b-46c6-83cb-7d11c9db3444 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:23.500854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:23.501874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.502393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.502914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.503754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] RBAC: Adding query filter params (name=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:23.504259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:23.505875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:23.511326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 358/717] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:23.523684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:23.528660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:23.614420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bb60e1-7997-485c-9ce7-b7d8a921e787 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:23.614711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:23.688654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:23.689272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.689549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.689820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.690592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=1d56ed96c2f546e59e473b1d62bf99eb) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:23.690893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:23.692816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148d9c59-3e0a-4039-a8c6-e84d8a44114d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:23.710899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc41112d-17c4-48eb-b37e-b87a448ea88b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:23.718671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 360/718] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:23 2022] GET /identity/v3/role_assignments?user.id=1d56ed96c2f546e59e473b1d62bf99eb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:23.792947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:23.793915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:23.794415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:23.794849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:23.796151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:23.796560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:23.798911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9206b2f1-1037-4977-8464-dc1606db5566 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:23.811478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 359/719] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:24.041553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa499b-f3e6-4200-b00c-283ea749c415 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:24.041896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa499b-f3e6-4200-b00c-283ea749c415 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:24.042074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa499b-f3e6-4200-b00c-283ea749c415 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:24.043563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 361/720] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:24.049654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:24.049937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:24.050238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:24.070657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:24.072044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:24.183973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 360/721] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:24.706929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa499b-f3e6-4200-b00c-283ea749c415 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:24.877379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa499b-f3e6-4200-b00c-283ea749c415 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:24.905866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:24.933103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:24.934077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:24.934559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:24.934997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:24.961897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:24.971008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:24.971951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:24.973356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 362/722] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:24.978490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:25.072718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:25.183926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb564fb-fbd7-4bcd-95a5-c97e5607beb7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:25.236777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:25.237285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.237631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.237900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.238608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:25.238892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:25.240251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f86ed24-e5ef-45f4-b5a6-d67692c675a2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:25.257687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 361/723] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:25.263235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:25.383433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e7826-8d3b-48df-a3c4-c279c3dac30a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.384105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e7826-8d3b-48df-a3c4-c279c3dac30a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.384727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895e7826-8d3b-48df-a3c4-c279c3dac30a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.387676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 362/724] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:25.396211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.396760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.397230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.404247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceca484-0e6c-47a1-8e71-b202f1fedbef None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:25.440220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:25.441827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:25.445401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:25.446050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.446364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.446638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.459277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:25.460912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:25.476760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:25.478233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:25.479426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:25.479426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:25.479426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:25.479426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 363/725] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:25.487437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:25.564721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:25.618051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 363/726] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:25.677940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908eb62-6046-4dbb-811b-858045f05c65 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:25.716087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:25.716759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.717000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.717246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.717902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:25.718226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:25.719637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5dc2b33-53d4-4900-91eb-198f0c2bb9ff None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:25.733047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 364/727] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:25.739613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:25.798859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:25.887324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e074-1778-4194-9ff9-7592c9989897 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.887819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e074-1778-4194-9ff9-7592c9989897 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.888253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70e074-1778-4194-9ff9-7592c9989897 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.890662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 365/728] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:25.897320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff769dc-4b94-40d4-99d4-9ca84d370d6e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:25.915204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.915667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.916089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.952470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:25.953577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:25.954260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:25.955223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:25.958338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:25.960272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:25.964483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:25.966120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:25.987092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:25.989915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:25.992803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:25.996933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 364/729] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:26.007473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.114572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 366/730] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:26.115752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.136528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.197428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903b2fc3-2a56-4cbe-8ca7-490d604abcc2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.200055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.289580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.290571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.291050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.291520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.292854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:26.293344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9716df71-e8f0-48bc-9e0a-c7e77bc12deb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.293458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:26.296913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:26.343746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.344782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.345267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.345869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.348517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 365/731] 10.222.0.91 () {56 vars in 1135 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:26.350601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.373138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:26.379740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:26.380751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:26.381598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:26.384435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 367/732] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:26.392881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.495659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.498021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53469ccf-1d28-4ff1-a6fe-8d1604c78a70 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.529865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d59a93-7d76-411b-ac30-94f5e54723ec None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.539921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.540607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.541362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.541362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.573046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.574059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.574546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.575057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.576525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.577654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:26.578925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:26.582557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:26.596843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 368/733] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:26.614283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.633173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:26.634794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:26.761289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b3289-3093-476c-b5d4-d78e9469e8d7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.762635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 366/734] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:26.763559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.805211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.805771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.806049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.806349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.815259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.817825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:26.819360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:26.840771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-bd33f8ba-ab3c-4603-b651-d72b866f8ead None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:26.849527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:26.851229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:26.852548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:26.852548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:26.852548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:26.852548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 369/735] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:26.852548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d143f-a110-48ac-a1cb-2b8d71dfca70 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.852548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d143f-a110-48ac-a1cb-2b8d71dfca70 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.852908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d143f-a110-48ac-a1cb-2b8d71dfca70 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.859046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 370/736] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:26.859046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203f395-6a87-46e2-868f-aab10da839e0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.859046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203f395-6a87-46e2-868f-aab10da839e0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.859046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a203f395-6a87-46e2-868f-aab10da839e0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.860357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 371/737] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:26.861737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203f395-6a87-46e2-868f-aab10da839e0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:26.898911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f07515-6c81-47a6-a678-660f95d4ff3d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:26.944121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:26.944784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:26.945070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:26.945345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:26.946119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:26.947552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:26.967620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 367/738] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:23:26 2022] POST /identity/v3/services => generated 251 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:26.968556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.033284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a203f395-6a87-46e2-868f-aab10da839e0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.096299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.097279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.097766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.098278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.115402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.117642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.143386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2459e032-5c3e-4a97-b507-767f107a0161 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-dfab5132-619e-4155-ab65-e3379c7a6a40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:27.144593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:27.145662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:27.147069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:27.147069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:27.147069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:27.148582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 372/739] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:27.151200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.151200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.151372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.178147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:27.179573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:27.182761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:27.200797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.202104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.202819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.203276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.204935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:27.205365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.207404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437f677c-cb2a-41c7-abd4-e567887bb5a3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.233558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 368/740] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:27.233558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.233558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.233558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.244387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:27.258130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:27.259859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:27.299443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 373/741] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:27.301209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.367507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 369/742] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 24 03:23:27.368941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.470221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5409fd-15ed-4288-af95-e95058492b81 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.511823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6559d02-3b5b-4416-9d9c-5b102abeb46f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.528271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:27.533401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.534665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.535699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.536192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.537448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:27.537984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.540921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.565485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 374/743] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:27.566757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.571190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.571738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.572212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.575065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.587072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.587072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:27.618751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:27.619750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:27.621088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 370/744] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:27.631143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.655022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:27.847920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463427b-34ed-4864-afc5-47d8fde8d2bc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.890213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.891752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.892248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.892753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.903529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.904419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeddbff6-b527-4658-ba12-4477c3231bfe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:27.905954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:27.924228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:27.925192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:27.926082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:27.930904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 375/745] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:27.932569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:27.954642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:27.955789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:27.956812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:27.957297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:27.958581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:27.959367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:27.961251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:27.984875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 371/746] 10.222.0.91 () {56 vars in 1149 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:27.987290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.029426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.077124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.096524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081bb1d-aabf-4d07-bc15-a8eecac16f53 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.142708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26822dd4-37c9-4747-bf22-7bed3cc5c23a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.158879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.160084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.160859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.161328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.180943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:28.188098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:28.189166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 376/747] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:28.190010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.195592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.196236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.196482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.196761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.197471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:28.197776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:28.199267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:28.213322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 372/748] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:28.214247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.351727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.358526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528c25f6-e178-4081-b2aa-239782e1d469 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.371859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec207501-9869-4517-948a-a5251d14f59d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.414391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.415325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.415786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.416238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.417325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:28.419730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:28.425841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.426834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.427281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.427726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.446241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 377/749] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:23:28 2022] POST /identity/v3/services => generated 240 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:28.446241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.471297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.531034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:28.532947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:28.616732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e772f46b-3035-42bf-97fe-fd90c199d207 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.642780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.657541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.658156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.658461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.658728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.659687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:28.659687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:28.660232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 373/750] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:28 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 24 03:23:28.661180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.661348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:28.679534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 378/751] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:28.679534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.782310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.786534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d57b8-4d20-4259-8aaf-304dce07360e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.833738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8ea364-59d9-4b24-b0e1-f9e7de1e2ff4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.834164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.834933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.835277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.835646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.836600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=c97977ebdd6f4f6da887c8a7215bda69) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:28.837000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:28.842241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:28.845702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 374/752] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:28 2022] GET /identity/v3/role_assignments?user.id=c97977ebdd6f4f6da887c8a7215bda69&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:28.846845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:28.930855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:28.951020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:28.951696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:28.951969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:28.952239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:28.972210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a593926-459d-4d0b-a198-a28ac2ff2c14 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:28.987124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:28.988613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.012500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.013158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.013440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.013715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.026891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.028441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:29.035237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:29.036504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:29.038470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:29.038470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:29.038470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:29.038470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 379/753] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:29.043725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:29.055137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:29.057081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:29.058224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:29.058224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:29.058224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:29.058224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 375/754] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:29.066255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.178394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f6d55-a76b-4f40-8c2c-6adf6df6a0c2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.209365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:29.227068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.227689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.227970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.228245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.228903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:29.229219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.230684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2261b2d8-49da-4194-b589-a6292bb33367 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.231134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a8dc20-5d35-4b1b-88ab-cd1596abd294 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.233980 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:29.246610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 380/755] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:29.248043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44689a-a3fb-49cb-81a5-c473daba4c00 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.248341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44689a-a3fb-49cb-81a5-c473daba4c00 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.248619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44689a-a3fb-49cb-81a5-c473daba4c00 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.250230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 381/756] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:29.254719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44689a-a3fb-49cb-81a5-c473daba4c00 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.287962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.288672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.288948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.289217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.289870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:29.290168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.291609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2509332-1911-49c0-9ae6-4c46c901bb25 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.306835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 376/757] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:29.308643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.309075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.309444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.329325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:29.330691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:29.385314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44689a-a3fb-49cb-81a5-c473daba4c00 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.415753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 377/758] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:29.416597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.467089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.467634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.467926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.468159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.481552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.486529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:29.510684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:29.512000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:29.513264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 382/759] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:29.518061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.607676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:29.647193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:29.664244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fdcc94-9b23-419a-a5eb-4bc38d557b2f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.736085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcc4db-b025-493b-94ea-7a887638b51a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.755262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.756144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.756595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.757025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.772097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.774347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.774524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.775191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.775552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.775893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.776655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:29.777018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.778816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:29.792633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 383/760] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:29.793616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:29.794486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:29.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:29.796842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:29.801954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 378/761] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:29.802889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:29.946691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:29.949180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce87719-eab4-40f6-8594-8cc6755f2f33 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.951728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d8ae0e-0f80-419c-b196-1838ac25e9ab None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:29.994116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:29.995356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:29.995787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:29.996235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:29.997134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:29.999051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.006802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.007972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.008544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.009108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:30.009788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:30.011014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:30.012089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:30.013031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=787f0e0c0d9d42c999c77c7eb75d4ceb) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:30.013362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.014466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 384/762] 10.222.0.91 () {56 vars in 1129 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:30.015551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.016094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.027785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 379/763] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:29 2022] GET /identity/v3/role_assignments?user.id=787f0e0c0d9d42c999c77c7eb75d4ceb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:30.028570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.101393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:30.152131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75547b57-5891-4f3d-8a0d-edd57c8e4bf6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:30.195748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.196792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.197255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.197884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.198884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:30.199405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.201065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.230435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 385/764] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:30.232952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.285358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:30.318101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6281c78e-c72e-4175-a25a-79032a12812a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:30.370336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cf0039-533e-489c-806d-91f2ba8abe91 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:30.413445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.414165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.414478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.414757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.427551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.429039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.445138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.445767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.446046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.446343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.446988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:30.447290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:30.447570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:30.448973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:30.450503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:30.450503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:30.450503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:30.450773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.458362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 386/765] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:30.462527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.468909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 380/766] 10.222.0.91 () {56 vars in 1124 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:30.475002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.603119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:30.617733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025d0790-3796-45d0-ae11-c7589b571940 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:30.703507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.704283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.704662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.704938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.705600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:30.706079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.707678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.728270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:30.731350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 387/767] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:30.741827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.768326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0f115-c674-4b17-82db-0ef8a757eaed None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:30.827512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:30.828131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:30.828409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:30.828679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:30.829349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:30.829650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:30.831061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:30.842362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 381/768] 10.222.0.91 () {56 vars in 1124 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:30.850728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:30.946101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb669395-ab8f-4104-9b43-1f56b1ed432b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.008758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:31.009640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.009996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.010383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.013839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:31.022752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:31.023214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10ede68-2957-47eb-ab93-997b717491f6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.024996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2855fd1c-3dc4-48a7-8833-4734631109a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:31.045606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:31.046622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:31.047565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:31.048468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 388/769] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:31.052922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd22d3-e768-4d71-9e64-1f2579033ec8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.052922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd22d3-e768-4d71-9e64-1f2579033ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.052922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acd22d3-e768-4d71-9e64-1f2579033ec8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.052922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 389/770] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:31.054718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd22d3-e768-4d71-9e64-1f2579033ec8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:31.071685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:31.072293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.072566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.072831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.073637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] RBAC: Adding query filter params (interface=public, service_id=ec4c76c2bce4401f98a4cfff9ee14806, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:31.073943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:31.075523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b867931-769c-46a0-85d0-7b50703e30f2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:31.089790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 382/771] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:23:30 2022] GET /identity/v3/endpoints?service_id=ec4c76c2bce4401f98a4cfff9ee14806&interface=public®ion_id=RegionOne => generated 194 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:31.089790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.089790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.089790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.122760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:31.125548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:31.129696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:31.216157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acd22d3-e768-4d71-9e64-1f2579033ec8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.242644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 383/772] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:31.244388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:31.294756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:31.296137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.297000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.297512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.299387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:31.300415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:31.303394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb47726-9a7d-414f-98c2-46bf19deb58b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:31.331396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 390/773] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:31.334104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9af4b7-f179-45ca-af74-35c88b243038 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.335276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9af4b7-f179-45ca-af74-35c88b243038 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.336041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9af4b7-f179-45ca-af74-35c88b243038 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.339602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 391/774] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:31.341246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9af4b7-f179-45ca-af74-35c88b243038 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:31.420934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df877f-3cd8-4da4-b3bf-70293ca56729 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.444809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:31.478824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ce1547-1bda-4b51-a4cb-4926088142b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:31.480278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce1547-1bda-4b51-a4cb-4926088142b1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.480761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce1547-1bda-4b51-a4cb-4926088142b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.481223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ce1547-1bda-4b51-a4cb-4926088142b1 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.482525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce1547-1bda-4b51-a4cb-4926088142b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:31.484196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ce1547-1bda-4b51-a4cb-4926088142b1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:31.498356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 384/775] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:23:31 2022] GET /identity/v3/services => generated 772 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:31.498356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.498356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.498356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.516297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:31.544629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:31.547947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:31.555410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9af4b7-f179-45ca-af74-35c88b243038 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.631798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:31.633089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.633726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.634367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/1d56ed96c2f546e59e473b1d62bf99eb/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.638398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=1d56ed96c2f546e59e473b1d62bf99eb, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:31.641609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:31.643859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 385/776] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:31.645355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:31.683504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 392/777] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:31 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/1d56ed96c2f546e59e473b1d62bf99eb/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:31.776814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb0344-2471-4b0a-838b-4ef84cff9d46 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.819242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:31.819902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:31.820182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:31.820454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:31.821055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:31.822738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:31.825174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:31.836452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:31.837537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:31.838429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 386/778] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:31 2022] GET /identity/v3/services/network => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:31.847389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:31.975771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c413fd-67bf-4db2-9e8c-d2702ef2da76 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:31.991144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:32.039450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.040077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.040248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.040514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.041165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] RBAC: Adding query filter params (name=network) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:32.041467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:32.042912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:32.050745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 393/779] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:32.058603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:32.086750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:32.191559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3826abe-f48d-4d63-9284-0337af2d94c5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:32.234395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.235381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.236372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.236887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.237705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:32.238270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:32.240409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:32.246734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 387/780] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:32.252151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:32.342159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7957a1ba-689a-4cb6-84a1-7a8793431007 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:32.386674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.387978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.388601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.389219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.409035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:32.414603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:32.427403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:32.428427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:32.429302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:32.432505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 394/781] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:32.438042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:32.523216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:32.524731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b1576-7f69-44e7-b23b-2d9813c92ade None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:32.592491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.593046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.593298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.593571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.594314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] RBAC: Adding query filter params (interface=public, service_id=9936096be2334b8f9a8f441b00772c69, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:32.595481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:32.597106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:32.603584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 388/782] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/endpoints?service_id=9936096be2334b8f9a8f441b00772c69&interface=public®ion_id=RegionOne => generated 194 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:32.609020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:32.616615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28521d28-980d-4483-b0e3-fb03e349aa5a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:32.670904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.671825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.672264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.672692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.673560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:32.674031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:32.675616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:32.694733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 395/783] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:32.699912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:32.738561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b3c8e-06ec-45b6-8549-5d65bb1a0334 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:32.766398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:32.780126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4634856-159a-4fe6-97c3-ce2ba7f9919c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.780872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634856-159a-4fe6-97c3-ce2ba7f9919c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.781073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634856-159a-4fe6-97c3-ce2ba7f9919c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.781339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4634856-159a-4fe6-97c3-ce2ba7f9919c None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.781962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4634856-159a-4fe6-97c3-ce2ba7f9919c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:32.783979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4634856-159a-4fe6-97c3-ce2ba7f9919c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:32.790483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 389/784] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:32.792081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca396fc-77be-4c37-b1a9-0905a1a87af9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.792197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca396fc-77be-4c37-b1a9-0905a1a87af9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.792404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca396fc-77be-4c37-b1a9-0905a1a87af9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.803373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 390/785] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:32.808143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.808468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.808832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.823468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:32.830038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e8399-c45a-43c9-b08f-b04a02419eaf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:32.847646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:32.848794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:32.898493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:32.899534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:32.900069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:32.900560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:32.927107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:32.928910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:32.938132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 391/786] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:32.958504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:32.959592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:32.960524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:32.960524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:32.960524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:32.968387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 396/787] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:32.973879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.150748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1695240-a3e0-4611-9d3e-bcddf71933f8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.189815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:33.207100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.207711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.207986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.208251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:33.208905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:33.209205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:33.210716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:33.224831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 392/788] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:33.230117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.256445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.375666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4edbf1-5543-49c2-b8ee-6ec18f71d886 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.415662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.416701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.417209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.417684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:33.427277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:33.429066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:33.441828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:33.443080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:33.444060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:33.446722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 397/789] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:33.454840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.480696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:33.557829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b9c79-246f-42d0-8833-9ee2571c3f2f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.581656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eaca1e-251b-4fd0-afef-bcb617781543 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.602968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.603612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.603888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.604158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:33.619775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:33.622670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.623520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.624022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.624521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:33.625475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:33.626607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:33.628669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:33.629387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:33.630373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:33.635170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 393/790] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:33.652053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.658058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 398/791] 10.222.0.91 () {56 vars in 1135 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:33.673880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.724254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:33.795337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:33.854631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af460a39-4122-4060-be74-b72eccb41e98 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.878355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e892b69-ed2b-4045-91a4-7f5afe7c13fb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:33.919382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.921075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.921075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.921289 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:33.922280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:33.923445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:33.925687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:33.942058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 394/792] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:33.953690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:33.975359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:33.976438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:33.976913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:33.977380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.005772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:34.007280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:34.041092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:34.042124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:34.043065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:34.043065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:34.043065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:34.049232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 399/793] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:34.051653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:34.234379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112e710-551e-45e9-8e33-937882fded06 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:34.239783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:34.303057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:34.303703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.303982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.304370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.322411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:34.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-00318948-2504-44e8-8c93-ee6e792903c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:34.351208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:34.352226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:34.353195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:34.353195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:34.353195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:34.353195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 395/794] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 400 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:34.355421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543b4bc-9bff-4ecf-81a9-452f6003977f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.355888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543b4bc-9bff-4ecf-81a9-452f6003977f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.356425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543b4bc-9bff-4ecf-81a9-452f6003977f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.359147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 396/795] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:34.360591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543b4bc-9bff-4ecf-81a9-452f6003977f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:34.383505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559bceee-6e28-4a29-bd36-833066f71686 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:34.418911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:34.441980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:34.442921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.443202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.443686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.444480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:34.444840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:34.446533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee3a04-5701-44c0-9712-782a69bb7231 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:34.462411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 400/796] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:34.467174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.467174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.467174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.484334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:34.486394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:34.508158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543b4bc-9bff-4ecf-81a9-452f6003977f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:34.551781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:34.552669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.553122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.553390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.554040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:34.554388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:34.555766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa515949-7b53-4f5f-94dc-74540ea5f4a0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:34.568679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 397/797] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:34.571105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45005592-6e59-4aec-ad44-26b716f08611 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.571393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45005592-6e59-4aec-ad44-26b716f08611 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.571813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45005592-6e59-4aec-ad44-26b716f08611 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.573213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 398/798] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:34.583277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45005592-6e59-4aec-ad44-26b716f08611 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:34.590247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 401/799] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:34.590247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:34.609744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:34.718714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45005592-6e59-4aec-ad44-26b716f08611 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:34.765715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:34.766282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.766561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.767412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.774735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:34.776245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:34.776531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342fcf-0444-4c63-a8d3-003f40c93d87 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4acaeb55-6f91-40bd-b267-ffef08bbebd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:34.789637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:34.790688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:34.791881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 399/800] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:34.793034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.795631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.795631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.804982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:34.820342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:34.822267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:34.834970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:34.836118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:34.836653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:34.837169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:34.854473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:34.857188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:34.874591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:34.875590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:34.876621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:34.882172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 402/801] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:23:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:34.883146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:34.932186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 400/802] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:34.932186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:34.936381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.041319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86d56da-c5b5-4f7f-b6d9-d2dc75380dc4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.077391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bddcf4-feb8-4ea6-b62f-a1e04eec9436 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.078509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.079277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.079657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.080037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.080864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:35.081981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.082909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.097173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 403/803] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:35.102966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.123836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.124770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.125235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.125651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.127214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:35.127811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.129592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.139826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.155393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 401/804] 10.222.0.91 () {56 vars in 1149 bytes} [Wed Aug 24 03:23:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:35.156652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.216444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edbb5f-2800-4372-9285-2a4ed1b315ef None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.237033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.260395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.261496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.261979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.262548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.263615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.266226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:35.279367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:35.281331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:35.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 404/805] 10.222.0.91 () {56 vars in 1129 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:35.288568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.293950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518b45e-dcdc-4fe3-9262-c86fdf42e25d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.348436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.349593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.350085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.350620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/c97977ebdd6f4f6da887c8a7215bda69/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.353569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=c97977ebdd6f4f6da887c8a7215bda69, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.355047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.371784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 402/806] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:35 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/c97977ebdd6f4f6da887c8a7215bda69/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:35.372726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.417761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.426666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8424fbdf-ddc8-41f4-976b-46896cf9b28b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.494548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.495754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.496127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.496354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.512368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.513894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.531214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676cb5b-3607-4010-ae84-c92eb609932e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:35.531916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:35.533350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:35.535048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:35.535048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:35.535048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:35.535048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 405/807] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:35.535048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.566643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.580173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a281b73-7f52-433f-9931-9ea92fd90120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.580809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281b73-7f52-433f-9931-9ea92fd90120 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.581315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281b73-7f52-433f-9931-9ea92fd90120 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.581873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281b73-7f52-433f-9931-9ea92fd90120 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.593507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a281b73-7f52-433f-9931-9ea92fd90120 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9a281b73-7f52-433f-9931-9ea92fd90120 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:35.598903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:35.599855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:35.600761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 403/808] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:35.601547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57030dfd-a570-4d0f-8337-84ab00a7337f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.601547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57030dfd-a570-4d0f-8337-84ab00a7337f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.601682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57030dfd-a570-4d0f-8337-84ab00a7337f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.606243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 404/809] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:35.606243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57030dfd-a570-4d0f-8337-84ab00a7337f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.679072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b767386-7869-476a-979d-b0fd35d08220 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.732083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.733075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57030dfd-a570-4d0f-8337-84ab00a7337f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.733263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.733710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.734166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.735462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:35.735972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.739656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.758370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 406/810] 10.222.0.91 () {56 vars in 1124 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:35.759211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.771923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.772527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.772878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.773063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.773712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:35.774009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.775425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1232b5-b9e8-40cd-b162-031967323d37 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:35.776226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.789031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 405/811] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:35.790345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.790636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.790917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.809793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:35.811051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:35.856792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:35.902235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 406/812] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:35.902235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:35.916189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07b32bc-016f-425b-a799-19a0b2efa5d4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:35.978876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:35.980076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:35.981413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:35.981992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:35.983691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:35.984400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:35.988286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.007964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 407/813] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:36.009909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.057516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd40e8d8-e31d-454b-934b-2fcace8f92fa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.105796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.107051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.107582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.108258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.109647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:36.110546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.114521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.128588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 407/814] 10.222.0.91 () {56 vars in 1124 bytes} [Wed Aug 24 03:23:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:36.130866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.137123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.174320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f3c2f7-75d0-48d8-b90a-4378589e624e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.193357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.229035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.230130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.230555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.230829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.237966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.239984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:36.252853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:36.254043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:36.255012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 408/815] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:36.255546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.287364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3438d3c7-f688-44a5-a0c2-ba11b61d4cbe None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.343742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.345640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.345640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.345890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.362325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.364037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.384638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaae993-680a-49d6-a8f5-18edd5cf484c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:36.388303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:36.389275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:36.390196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:36.390196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:36.390196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:36.390196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 408/816] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:36.390430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.431797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.432654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.432654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.433008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.433579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.434982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.441835 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.470217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 409/817] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:23:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:36.470217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.530323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.549965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2967de-e732-45bb-9849-323169c59ac5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.604236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.605728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.606257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.606743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.607562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.609280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.614529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f99502-b75f-4f03-ba80-38edc1c292f8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:36.620607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:36.621946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 24 03:23:36.623187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:36.625809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 409/818] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/services/network => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:36.627475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.655615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.656321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.656715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.657205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.658527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:36.658877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.660353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.675731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 410/819] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:36.677075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.749973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.771598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bbde84-65da-4be8-934c-04d80fffe6d1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.819588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d20a3f-81e3-4da7-b9d6-4b1c64fa9efe None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.820427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.821051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.821328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.821595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.822322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:36.822752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.824680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.846567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 410/820] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:36.847498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.859571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:36.862709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:36.863611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:36.864086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:36.864617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:36.865547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] RBAC: Adding query filter params (name=network) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:36.866054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:36.868541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:36.874377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 411/821] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:36.875613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:36.973524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f22435-d273-4692-be77-6b44eea4c4a8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:36.996632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60a0344-2001-4902-952f-db16eb8b50e9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:37.011249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.011913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.012188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.012470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/787f0e0c0d9d42c999c77c7eb75d4ceb/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.022689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:37.032886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=787f0e0c0d9d42c999c77c7eb75d4ceb, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.034767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:37.050326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 411/822] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:36 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/787f0e0c0d9d42c999c77c7eb75d4ceb/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:23:37.051396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:37.068167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.068895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.069173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.069447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.080317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.082381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:37.099428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:37.100486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:23:37.101446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:37.105815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 412/823] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:23:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:37.107475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:37.209173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:37.244169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1163233-e6a6-4300-bcc5-1efdb4c1355e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:37.292225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.292850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.293126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.293255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b552ff-7b5b-47bb-ae64-99c638e20a64 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:37.293436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.294131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:37.294465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.296228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:37.330341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 413/824] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:23:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:37.333119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:37.373347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:37.383345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.384205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.384579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.384940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.385913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] RBAC: Adding query filter params (type=network) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:37.386377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.388442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:37.405497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 412/825] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:37.411847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:37.461365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5689287-5736-4a35-8a1b-4af09551d184 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:37.525736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.526429 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.526814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.527092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.537165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b46dc4b-06e8-4e07-9a54-1a91b5b954aa None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:37.561605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:37.574295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0e94da-a5b7-428e-81d9-dd665e49e86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.574914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e94da-a5b7-428e-81d9-dd665e49e86e None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.575190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e94da-a5b7-428e-81d9-dd665e49e86e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.575461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0e94da-a5b7-428e-81d9-dd665e49e86e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.576131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e94da-a5b7-428e-81d9-dd665e49e86e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.577540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0e94da-a5b7-428e-81d9-dd665e49e86e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:37.601674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 413/826] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:23:37 2022] POST /identity/v3/endpoints => generated 337 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:23:37.603213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c751e6-f241-429a-a99d-05a351df9123 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.603507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c751e6-f241-429a-a99d-05a351df9123 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.603832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c751e6-f241-429a-a99d-05a351df9123 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.609673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 414/827] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:37.611588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.611870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.612160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.623889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.628011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:37.633821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:37.634937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:37.677455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:37.721978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 415/828] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:37 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:37.797644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 414/829] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:23:37.805737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:37.935334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1199cc8e-8312-4ef2-8b30-f49e95ab9696 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:37.938549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:37.979914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:37.980466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:37.980739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:37.981003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:37.992305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:37.993790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:38.009577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:38.010683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:38.011578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:38.011578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:38.011578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:38.015477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 416/830] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:38.017410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.115195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.167650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b6261-bd6f-4944-ac0a-bca0409e9b26 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.211734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:38.220152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.220978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.221365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.221680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.223243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:38.223679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.225992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.250446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 415/831] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:38.253530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.261143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6251a-1452-4844-a2d2-733faac55e16 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.300257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.300882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.301159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.301422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.312678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:38.318507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:38.320660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:23:38.321737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:38.324567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 417/832] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:38.325541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.346728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:38.379432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab06595-b1e3-4be9-a1a5-c20658426ebd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.418398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.419403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.419896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.420428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.428612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.430410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.442438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a109e-e0d6-44fa-babe-a04798297162 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:38.444348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:38.445255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:38.446165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:38.453181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 416/833] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:38.455585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.482295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.482890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.483161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.483433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.484101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:38.484404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.485824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.503483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 418/834] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:38.504611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.519469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:38.578368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88442c6-ad37-491d-9a21-9963b121d672 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.607870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:38.622376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.623456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.623949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.624422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.625456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:38.625976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.628332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.636894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aafd2b-6a97-4805-87bf-5c3fd672c133 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.648988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 417/835] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:38.650957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.689413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.690119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.690635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.690910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.705343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.707479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:38.725669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:38.726762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:38.727648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:38.727648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:38.727648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:38.729896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 419/836] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:38.736399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:38.793502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0bfbc1-b18c-4918-a4b2-f416717a9421 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.813296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:38.843247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.843885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.844163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.844418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.845916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=1d56ed96c2f546e59e473b1d62bf99eb) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:38.846371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.848963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073605ea-96ec-49ad-bd11-c8b94b9c4d61 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.857266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 418/837] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/role_assignments?user.id=1d56ed96c2f546e59e473b1d62bf99eb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:38.875492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b892f3-4c8f-4297-adfb-4f6a8442f18f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.876380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b892f3-4c8f-4297-adfb-4f6a8442f18f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.877061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b892f3-4c8f-4297-adfb-4f6a8442f18f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.880142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 419/838] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:38.886850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.887755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.888423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.896237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e3cef0-ddd5-43dc-9d2f-80da747cac50 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:38.918560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:38.920938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:38.935951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:38.943200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:38.943690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:38.943940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:38.944211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:38.945244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:38.945557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:38.947032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:38.964720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 420/839] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:38.971052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.025288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 420/840] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:38 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:39.097879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f7915-67cc-4d39-988e-83c0be2d6408 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:39.141145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:39.141805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.142709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.143166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] PATH_INFO: `/v3/users/swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.152887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:39.154388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:39.169060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:39.171325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:39.173330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 421/841] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:23:38 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:39.177036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.303151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0176c539-ee10-4c97-9ded-c28111791677 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:39.311874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:39.330440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.346879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:39.348280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.348792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.349525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.351120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:39.351821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:39.353953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:39.380307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 421/842] 10.222.0.91 () {56 vars in 1135 bytes} [Wed Aug 24 03:23:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:39.389701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.398953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:39.482559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8efbc0f-528f-47df-9521-808744a61ddc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:39.527682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4035c-845b-4921-a914-3743e36e6ef0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:39.537083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:39.537724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.537999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.538292 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.539254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:39.541222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-82a25482-1fac-416d-8ee6-aefea51069f4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:39.552204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:39.553350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:39.554349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 422/843] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:39 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:39.556059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ec4a-c692-4ce6-bd2f-8aadac68ba98 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.556059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ec4a-c692-4ce6-bd2f-8aadac68ba98 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.556059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ec4a-c692-4ce6-bd2f-8aadac68ba98 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.557890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 423/844] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:39.560181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ec4a-c692-4ce6-bd2f-8aadac68ba98 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.580246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:39.581134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.581417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.581687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.644806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:39.699149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ec4a-c692-4ce6-bd2f-8aadac68ba98 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:39.713198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:39.716070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:39.717979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:39.749225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:39.750122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.750647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.750938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.752148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:39.752446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:39.754825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cab5d2-f8e2-4232-8fc0-38782b7e4842 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:39.768483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 424/845] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:39.768483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:39.768483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:39.768483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:39.789551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:39.790981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:39.886798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 422/846] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:39 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:23:39.886798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.910306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 425/847] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:39 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:39.910306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:39.984617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:40.046581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22115f-68c6-4977-ac12-6548e41eac6d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.081094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e170be9-3bf6-487b-9667-84e383358bf5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.096420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:40.099037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15468490-875a-4c69-84df-430bace68421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.099867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15468490-875a-4c69-84df-430bace68421 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.100051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15468490-875a-4c69-84df-430bace68421 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.100368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15468490-875a-4c69-84df-430bace68421 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.101074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15468490-875a-4c69-84df-430bace68421 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:40.101775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15468490-875a-4c69-84df-430bace68421 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.106926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15468490-875a-4c69-84df-430bace68421 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.115043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 423/848] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:40.140610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.141563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.142580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.143196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.156654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.159093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:40.177414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:40.178631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:40.179707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:40.179707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:40.179707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:40.181692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 426/849] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:40.188000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15468490-875a-4c69-84df-430bace68421 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.248695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.365360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15468490-875a-4c69-84df-430bace68421 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.403946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87b1d7a-acb2-46f9-8b22-de634db3b522 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.410602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:40.415712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.416727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.417249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.418399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.419798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:40.420172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.424035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.456043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.456681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.457166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.457329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.457780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 424/850] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:40.463461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.470647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:40.476641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:40.477673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:23:40.478580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:40.481440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 427/851] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:40.488611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.528759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:40.621963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7104ec94-c8a5-4a9f-9c32-aede8f37ae23 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.651690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a549ce5-a1ef-408e-8512-77b7734c75d4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.673131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.675009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.675464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.675839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.693948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.696905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.708718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.709392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.709525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.709799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.710471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:40.710795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.712346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:40.714931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:40.715871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:40.716796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:40.719463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 425/852] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:40.725416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.730436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 428/853] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:40.732381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.798283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:40.868597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:40.875477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c89b47-06be-4ac1-a0b8-796caf6f534e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.895624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b72e3-ea5b-48bc-ace3-fc59502a7902 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:40.929351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.930870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.931741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.932588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.933889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:40.934529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.936844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.943480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:40.944247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:40.944635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:40.944913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:40.957524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 426/854] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:40.960131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:40.962873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:40.963645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:40.983314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:40.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:40.985481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:40.985481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:40.985481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:40.992927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 429/855] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:40.994168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.108541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6145a3ea-3fe8-4253-bc5e-e031b52e284e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:41.122155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:41.140721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a080bed4-2b22-46d7-88b9-fb2a0081ba9e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:41.159663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:41.160714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.161009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.161285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.162388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=c97977ebdd6f4f6da887c8a7215bda69) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:41.162710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:41.164431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad1b6d5-3902-4001-b840-bd8568b31595 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:41.174465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 427/856] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/role_assignments?user.id=c97977ebdd6f4f6da887c8a7215bda69&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:41.195802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:41.196810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.197310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.197767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.198702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:41.199225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:41.201130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:41.203825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:41.217172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b87ef-9a30-4493-9bb2-34909ef773c8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.217352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b87ef-9a30-4493-9bb2-34909ef773c8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.217622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469b87ef-9a30-4493-9bb2-34909ef773c8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.219413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 428/857] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:41.221202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 430/858] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:41.228849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.230946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.230946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.232389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.254466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:41.255834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:41.373142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 429/859] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:41 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:41.390502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baf0d5e-f90f-4222-b3dc-3edb9c02b753 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:41.452589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:41.453237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.453545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.453786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.466560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:41.469096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:41.485439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:41.486976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:41.488885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 431/860] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:23:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:41.497229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.625223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698e63-2068-447f-951b-2e59cf441062 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:41.627198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:41.666569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:41.667187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.667472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.667747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.668485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:41.668790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:41.670220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:41.690900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 430/861] 10.222.0.91 () {56 vars in 1149 bytes} [Wed Aug 24 03:23:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:41.696630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.709472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.761265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:41.822714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d53995-b6f6-4d43-89bb-a20fa55dfa5a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:41.831370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3d4d36-dd6a-4e43-98df-943470a0b4cf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:41.860231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:41.861745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.862363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.862841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.873203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:41.874056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:41.874176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:41.874403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:41.874973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:41.877275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:41.888451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:41.889497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:41.890468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:41.897890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 431/862] 10.222.0.91 () {56 vars in 1132 bytes} [Wed Aug 24 03:23:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:41.899473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:41.930630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:41.955985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:41.961076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.031959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.033773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1be52d5-3e23-4c5b-b0bf-02b00dcf5e4f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.084706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.085541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.085920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.086353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.087244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:42.087655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.089334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.102235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 432/863] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:42.110147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.121210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 432/864] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:41 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:23:42.122264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.243270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.259814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabdef07-46f4-4021-b48c-910176480c77 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.267823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab27e3-ae4b-463d-af38-27d26fdb3a3a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.315992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.316699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.317393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.317393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.318077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:42.318352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.319743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a17332e-78fd-4dc0-a23f-5109be491aa3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.326357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 433/865] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:42.326357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ff39-48c1-4563-9afc-162952e6e8d4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.326776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ff39-48c1-4563-9afc-162952e6e8d4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.326776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5ff39-48c1-4563-9afc-162952e6e8d4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.328232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 434/866] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:42.334483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.334743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.335014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.340692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.341734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.342413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.343024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.359098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.359873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:42.361036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.361841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:42.387669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:42.388783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:42.389734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:42.389734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:42.389734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:42.391657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 433/867] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:42.397590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.412028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.478395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 435/868] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:42 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:42.562774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64937a8-7c73-4f69-bbb7-27e78f0f9805 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.600942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.601480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.601758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.602026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.602852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:42.603155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.604560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.620001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 434/869] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:42.625435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.745714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.770646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.773649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af62ff26-e0a7-4d15-a7e0-8e5100018efb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.820776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.821682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.821958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.822317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.832797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.834917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:42.847837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:42.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:42.850261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 436/870] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:42.854837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:42.861063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:42.901805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da734d4-f673-4f70-aa28-d4fae9d1d7ea None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.942792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:42.944210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:42.945043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:42.945486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:42.946261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:42.947919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:42.966339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea62ccb-b000-4548-bc8d-63831463dcd1 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:42.969442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 435/871] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:23:42 2022] POST /identity/v3/services => generated 251 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:43.039927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:43.040892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:43.041483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:43.041909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:43.043066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:43.043655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:43.046092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aeb91b-22ff-4944-b0c6-cc4f082241ee None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:43.057491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:43.069219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 437/872] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:43.078320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:43.240428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9ad41-1cb2-4673-9435-8fd4b1973df3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:43.271940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:43.285940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:43.288399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:43.289556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:43.290549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:43.292438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=787f0e0c0d9d42c999c77c7eb75d4ceb) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:43.292983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:43.295761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f36d4-91f4-469b-8750-228ea8c3b1e3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:43.305413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 436/873] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:43 2022] GET /identity/v3/role_assignments?user.id=787f0e0c0d9d42c999c77c7eb75d4ceb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:43.597112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe547-92f5-4f20-8141-f53f403eb015 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:43.597697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe547-92f5-4f20-8141-f53f403eb015 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:43.598068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fe547-92f5-4f20-8141-f53f403eb015 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:43.600075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 438/874] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:43.608193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:43.608506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:43.608736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:43.629814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:43.631065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:43.711322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 437/875] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:43 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:44.045756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe547-92f5-4f20-8141-f53f403eb015 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:44.191272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fe547-92f5-4f20-8141-f53f403eb015 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:44.236718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:44.237810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.238318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.238604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.239206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:44.240584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:44.253850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:44.254978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:44.255856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:44.256618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 439/876] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:44.262458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:44.395298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78cadf-60a6-4b6e-84bc-0e20247ba263 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:44.419926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:44.436167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:44.436784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.437059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.437325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.437964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:44.438291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:44.439726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb65b13-9956-4244-b4e5-4376dfc075cd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:44.446531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 438/877] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:44.455345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:44.567286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:44.620179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2877ec-a561-45c9-aae9-88e519af35e6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.620610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2877ec-a561-45c9-aae9-88e519af35e6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.620863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2877ec-a561-45c9-aae9-88e519af35e6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.622515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 439/878] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:44.631118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c97c-241d-4ff9-a053-a1923ce35b21 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.631860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c97c-241d-4ff9-a053-a1923ce35b21 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.632575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f7c97c-241d-4ff9-a053-a1923ce35b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.638403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d81c42c-c64e-4169-b1fb-ba708731e267 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:44.665110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a8f7c97c-241d-4ff9-a053-a1923ce35b21 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:44.665968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a8f7c97c-241d-4ff9-a053-a1923ce35b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:44.704992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:44.705866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:44.706322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:44.706910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:44.708791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:44.709434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:44.711715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:44.719840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 440/879] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:44.778367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 440/880] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:44 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:45.116013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:45.175578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e975633-7124-444c-a005-2d31a1abaeb9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.175935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e975633-7124-444c-a005-2d31a1abaeb9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.176144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e975633-7124-444c-a005-2d31a1abaeb9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.177697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 441/881] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:45.186239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.186239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.186239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.207950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:45.209677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:45.247233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb996e0d-de05-4d44-a940-0b8c1a0d270c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:45.281831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:45.292581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5bf10-8dfd-4a11-b7de-83cdc7fc7a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:45.293737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5bf10-8dfd-4a11-b7de-83cdc7fc7a01 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.294177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5bf10-8dfd-4a11-b7de-83cdc7fc7a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.294625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5bf10-8dfd-4a11-b7de-83cdc7fc7a01 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.295428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5bf10-8dfd-4a11-b7de-83cdc7fc7a01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:45.296935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d5bf10-8dfd-4a11-b7de-83cdc7fc7a01 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:45.322217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 442/882] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:45 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:45.325686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 441/883] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:23:45 2022] POST /identity/v3/services => generated 269 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:45.638445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:45.691091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c1b7d-5b31-443c-bea9-e2542640c2b5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.691431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c1b7d-5b31-443c-bea9-e2542640c2b5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.691649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c1b7d-5b31-443c-bea9-e2542640c2b5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.693244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 442/884] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:45.694412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:45.698949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.699270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.699505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.723616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:45.724902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:45.774750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203de7c5-1186-4428-9f0f-28c622a69917 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:45.830586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 443/885] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:45 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:45.840029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:45.841172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:45.841896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:45.842644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:45.843953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:45.846773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:45.859805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:45.861343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:45.863055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:45.872286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 443/886] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:45.889515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.052922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383b78b-c9c5-4b6f-8c11-eced9d0e1943 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.068158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:46.098989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.100144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.100588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.100859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.101825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:46.102131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:46.109592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:46.116244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 444/887] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:46.124888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.225662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.247340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02be287b-7c77-433b-99d2-256245409ce6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.258541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:46.287392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.288635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.289213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.289774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.290646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:46.291248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:46.292830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:46.299641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 444/888] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:46.306412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.352557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf39f-2679-4a56-9653-217ffed773f3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.400606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.402144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.403045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.403657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.418950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:46.426827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:46.427859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:23:46.428711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:46.431634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 445/889] 10.222.0.91 () {56 vars in 1120 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:46.436884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5784a98c-55aa-44f7-b654-fcd70203ea72 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.442468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.500145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:46.510236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.511013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.511325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.511540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.512260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] RBAC: Adding query filter params (interface=public, service_id=2f76f800448c4152b67429a937af4e59, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:46.512566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:46.513959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:46.520026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 445/890] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/endpoints?service_id=2f76f800448c4152b67429a937af4e59&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:46.526805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.584904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:46.603832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec4d28-3e88-4306-8312-c160f8ad5480 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.653830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.654493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.654946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.655387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.656222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:46.656691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:46.658404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:46.668657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 446/891] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:46.673968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:46.694122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bdcf85-9536-41d1-9acc-1d379f6852bb None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.744323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.744943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.745275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.745541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.746434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:46.748037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:46.762591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 446/892] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/services => generated 1268 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:23:46.832626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:46.837017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e0cc35-9c26-4bbb-bf68-6bba54308861 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:46.883199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:46.884028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:46.884416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:46.884794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:46.899608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:46.901275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3c6326c3-f085-4e3d-9e47-e1fd8e7b98ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:46.919113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:46.920139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:46.921036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:46.921036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:46.921036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:46.921036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 447/893] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:46.928031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:47.002357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7be111-9031-4178-a2e6-f073ac264494 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.002357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7be111-9031-4178-a2e6-f073ac264494 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.002876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7be111-9031-4178-a2e6-f073ac264494 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.003973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 448/894] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:47.010879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.011754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.012301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.037256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:47.039533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:47.071902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:47.075444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53259e75-8c83-41e5-9b74-41ac3b2cac58 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:47.121909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:47.122829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.123392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.123990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.124869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:47.125376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:47.127979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:47.145176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 447/896] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:47.146328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 449/896] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:47 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:47.150798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:47.295858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816c43ad-1027-4ce0-a704-a11af05c08bc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:47.335330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:47.336087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.336361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.336684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.347259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:47.349046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:47.362600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:47.364544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:47.366482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 448/897] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:23:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:47.371728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:47.480427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:47.503771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:47.510528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237ba16-7bf8-4c33-9122-d61ad7706621 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:47.542222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:47.583733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:47.585206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.585949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.586618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.588256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:47.589121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:47.592219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:47.619084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 450/898] 10.222.0.91 () {56 vars in 1149 bytes} [Wed Aug 24 03:23:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:47.624801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:47.660357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca9ebe7-567c-4fe2-93e3-265764cff32a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:47.706167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8192feb-2a7d-4455-bc89-a19957b6394e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:47.707205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192feb-2a7d-4455-bc89-a19957b6394e None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.707699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192feb-2a7d-4455-bc89-a19957b6394e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.708182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8192feb-2a7d-4455-bc89-a19957b6394e None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.709356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8192feb-2a7d-4455-bc89-a19957b6394e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:47.711773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8192feb-2a7d-4455-bc89-a19957b6394e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:47.733433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 449/899] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:23:47 2022] POST /identity/v3/services => generated 241 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:47.735488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb426fb-5a30-4a72-bb8d-6efcad03cd6b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.736127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb426fb-5a30-4a72-bb8d-6efcad03cd6b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.736584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb426fb-5a30-4a72-bb8d-6efcad03cd6b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.738319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 450/900] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:47.744901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.745538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.745993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.769168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:47.770807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:47.783314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cff8543-3bfb-4d59-b72e-0beb05ffec6c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:47.849896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:47.850900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:47.851359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:47.851835 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:47.886637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:47.916859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 451/901] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:47.945272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:47.947654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.084442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 451/902] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:23:47 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:23:48.110981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.149263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:48.257409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.288831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b5881-875b-4a03-9059-3c4eeb2c29c6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:48.330637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:48.335036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:48.335788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:48.336117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:48.336412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:48.351791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:48.354509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:48.381612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:48.382621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:48.383542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:48.383542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:48.383542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:48.388975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 452/903] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:48.390574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.430817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f58d9e-e3e1-46e2-a051-4363c42cfbb0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:48.472234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:48.472954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:48.473150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:48.473421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:48.474019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:48.475440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.480411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:48.485432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:48.486993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:48.487897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 452/904] 10.222.0.91 () {56 vars in 1132 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:48.493097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.540756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b6698-dcda-43e7-a7ac-8566571812f3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:48.589083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:48.590489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:48.590951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:48.591678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:48.592522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:48.593027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:48.594923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.610387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 453/905] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:48.617407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.665824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:48.676207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90b2c3-5b9a-4d5c-8126-84ab59463965 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:48.727617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:48.728358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:48.728700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:48.729041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:48.729755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:48.730058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:48.731852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.743626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 453/906] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:48.750350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.795501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb7d6ec-15d8-44b6-931a-1fdadaa7c355 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:48.831571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:48.852731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:48.853790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:48.854456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:48.854883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:48.868939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:48.872387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:48.891441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:48.893427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:48.895302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:48.903396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 454/907] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:48.904982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:48.953569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2270c60f-d7e5-4754-b486-a9524b9a0566 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:49.018637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:49.019598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.020085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.020552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.021648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:49.022237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:49.024602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9344e7b-fd26-4908-b9c8-46a0fb25a893 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:49.038779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 454/908] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:49.041827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2db9c-f5bb-44d2-8e74-45e991d35bbe None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.042375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2db9c-f5bb-44d2-8e74-45e991d35bbe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.043278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d2db9c-f5bb-44d2-8e74-45e991d35bbe None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.045672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 455/909] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:49.047196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2db9c-f5bb-44d2-8e74-45e991d35bbe None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:49.088801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbb57d2-5139-4c61-978f-64ffd194ee5d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:49.101393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:49.149163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:49.149782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.150063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.150363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.151152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:49.151438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:49.153064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061ce3e8-e134-4e0f-8b58-438cf19a8e36 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:49.167818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 455/910] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:49.168975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.169335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.169607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.190579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:49.192877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:49.229982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d2db9c-f5bb-44d2-8e74-45e991d35bbe None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:49.261965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:49.277327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:49.278125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.278320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.278734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.279397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] RBAC: Adding query filter params (interface=public, service_id=5a1fb56ebfb74bd0bc1e046b63854958, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:49.279704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:49.281073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:49.303484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 456/911] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:23:49 2022] GET /identity/v3/endpoints?service_id=5a1fb56ebfb74bd0bc1e046b63854958&interface=public®ion_id=RegionOne => generated 194 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:49.303484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:49.323530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 456/912] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:49.324839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:49.439921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:49.456004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab2c88-59f3-4ace-83cb-fd58043a0e61 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:49.460449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195068b-4162-441a-ab72-ba358b1de41a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:49.497106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:49.498107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.498659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.499161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.499969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:49.502099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:49.508927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:49.509547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.509858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.510112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.511060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] RBAC: Adding query filter params (role.id=d864158e9f14416d9b63a27f1796d25c, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=787f0e0c0d9d42c999c77c7eb75d4ceb) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:49.511355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:49.518222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:49.519100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 457/913] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:23:49 2022] GET /identity/v3/services => generated 1497 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:23:49.526130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 457/914] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:23:49 2022] GET /identity/v3/role_assignments?user.id=787f0e0c0d9d42c999c77c7eb75d4ceb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=d864158e9f14416d9b63a27f1796d25c => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:49.692610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:49.747637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:49.842329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4761-8aad-4e6a-a9d3-4ad737fe1338 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:49.887148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:49.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.888439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.888714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.889406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:49.891656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-dd5d24b4-d941-476c-ad80-af890adf048a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:49.903228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:49.904309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:49.905334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 458/915] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:49 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:49.913705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:49.991157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d159b55-bb56-4871-afcb-7c724eba7df4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.991439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d159b55-bb56-4871-afcb-7c724eba7df4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.991729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d159b55-bb56-4871-afcb-7c724eba7df4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:49.997247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 459/916] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:49.999291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:49.999510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:49.999777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.026489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:50.027746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:50.066834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946edc3-f429-46df-8d31-eaa1c5e80822 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:50.118813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:50.119724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.120165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.120671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.121464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:50.121936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:50.123568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:50.130550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 458/917] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:50.139402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:50.141021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 460/918] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:50.169709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:50.293141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86094483-81be-4066-9d9e-11955fb90aca None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:50.338316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:50.339311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.339822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.340325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.341183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:50.341725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:50.343355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:50.349908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 459/919] 10.222.0.91 () {56 vars in 1114 bytes} [Wed Aug 24 03:23:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:50.355498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:50.408611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:50.446468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:50.513164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbac3c53-4fac-4c6a-b5f3-e1963ff7e202 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:50.564384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:50.565606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.566366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.566961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.568117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:50.570070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:50.590062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:50.603724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 461/920] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:23:50 2022] POST /identity/v3/endpoints => generated 346 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:50.631151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a24b3-ac1d-4084-8316-1b8e8098d1e8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:50.691492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:50.692271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.692632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.693031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.693939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:50.695636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:50.706501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:50.707605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:50.708555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 460/921] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:50.718060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:50.858644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:50.879033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c62f9-6b99-4ccb-871a-1cfe5b451c72 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:50.926849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:50.927535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:50.927823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:50.928087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:50.928729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:50.929032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:50.930810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:50.937002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 462/922] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:50.943557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:51.081200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5f7d55-c0e1-42e7-80b9-878ad9a28498 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:51.123875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:51.124417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:51.124703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:51.124964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:51.125592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:51.125888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:51.127856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:51.133424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 461/923] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:51.140164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:51.268825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b86363-8d3a-4750-ab1a-8d0ae2fe2324 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:51.315583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:51.316154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:51.316438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:51.316711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:51.317423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] RBAC: Adding query filter params (interface=public, service_id=2900434579b44c71855033084d25a71f, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:51.317735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:51.319148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9dee3f-db74-4937-accc-5eaa7782f2a8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:51.330398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 463/924] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:23:51 2022] GET /identity/v3/endpoints?service_id=2900434579b44c71855033084d25a71f&interface=public®ion_id=RegionOne => generated 194 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:51.336582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:51.348720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:51.475838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793dab21-8ec3-4c14-ba66-108be71bbcc0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:51.515675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03002ebf-3ffc-4149-9e66-bb2f6208202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:51.516300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03002ebf-3ffc-4149-9e66-bb2f6208202a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:51.516578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03002ebf-3ffc-4149-9e66-bb2f6208202a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:51.516852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03002ebf-3ffc-4149-9e66-bb2f6208202a None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:51.517510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03002ebf-3ffc-4149-9e66-bb2f6208202a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:51.519602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03002ebf-3ffc-4149-9e66-bb2f6208202a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:51.534391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 462/925] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:23:51 2022] GET /identity/v3/services => generated 1497 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:23:51.818630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad86fc-c038-44de-a10d-f72282bcb905 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:51.819131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad86fc-c038-44de-a10d-f72282bcb905 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:51.819400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ad86fc-c038-44de-a10d-f72282bcb905 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:51.821000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 464/926] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:51.827117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bf1c-78c7-45e1-8448-833a72fa00c3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:51.827758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bf1c-78c7-45e1-8448-833a72fa00c3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:51.828240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf8bf1c-78c7-45e1-8448-833a72fa00c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:51.848704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2bf8bf1c-78c7-45e1-8448-833a72fa00c3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:51.850327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2bf8bf1c-78c7-45e1-8448-833a72fa00c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:51.926753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:51.932471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 463/927] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:52.068169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dc99-1e5d-45a8-8265-72ee04f9468e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:52.068580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dc99-1e5d-45a8-8265-72ee04f9468e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:52.069058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086dc99-1e5d-45a8-8265-72ee04f9468e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:52.071264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 465/928] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:52.077828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:52.078265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:52.078628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:52.110370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:52.112658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:52.201625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 464/929] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:52.257522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086dc99-1e5d-45a8-8265-72ee04f9468e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:52.279272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:52.388094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086dc99-1e5d-45a8-8265-72ee04f9468e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:52.426397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:52.427094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:52.427373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:52.427659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:52.428312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:52.429795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:52.439891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:52.440940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 24 03:23:52.441846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:52.446070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 466/930] 10.222.0.91 () {56 vars in 1132 bytes} [Wed Aug 24 03:23:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:52.447938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:52.582372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dca5fc-3750-470d-949c-4ca6d1ef6849 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:52.582821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:52.619107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:52.628626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:52.629753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:52.630244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:52.631140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:52.631977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:52.632732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:52.634344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:52.646777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 465/931] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:52.648630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:52.729870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:52.774085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3653f14-a789-44f5-841a-419d6e2a35b2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:52.794033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a547ae-304c-49d3-8590-652ef07e82a8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:52.818509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:52.819156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:52.819685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:52.819778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:52.840843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:52.841762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:52.842262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:52.842812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:52.842998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:52.843911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:52.844433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:52.846111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:52.851440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:52.853641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:23:52.855284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:52.858832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 466/932] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:52.859040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 467/933] 10.222.0.91 () {56 vars in 1120 bytes} [Wed Aug 24 03:23:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:52.862742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:52.863293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:52.965672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:52.993388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4613a6d-407c-4245-b474-c65adf80412f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:53.039549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:53.040157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.040430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.040729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.041358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:53.041656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:53.043140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:53.049837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:53.061824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 468/934] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:23:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:53.062052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2139fbba-6fd6-4087-8a46-0fd8a946cb5f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:53.068533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:53.132725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d072b2fb-6e8b-48e7-9157-c580dad6c9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:53.133897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072b2fb-6e8b-48e7-9157-c580dad6c9ca None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.134557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072b2fb-6e8b-48e7-9157-c580dad6c9ca None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.135300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d072b2fb-6e8b-48e7-9157-c580dad6c9ca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.136672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072b2fb-6e8b-48e7-9157-c580dad6c9ca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:53.138542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d072b2fb-6e8b-48e7-9157-c580dad6c9ca None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:53.170226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 467/935] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:23:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:53.170226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e66ae-9e22-4ff9-931a-90fc2d492c78 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.170226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e66ae-9e22-4ff9-931a-90fc2d492c78 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.170226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7e66ae-9e22-4ff9-931a-90fc2d492c78 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.170226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 468/936] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:53.179590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.180065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.181173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.204655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:53.205669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:53.305157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33424e97-c40f-483b-bad6-86afe9f8097f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:53.362245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 469/937] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:53.375336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:53.376165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.376603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.377033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.395707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:53.397520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:53.415438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:53.417003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:53.418009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:53.418009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:53.418009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:53.418009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 469/938] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:53.422988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:53.468053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:53.542606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aa7289-73ab-45a3-b9ca-6a8dd4159f9c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:53.583309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:53.584150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.584832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.585698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.587692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:53.588007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:53.589473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:53.605784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 470/939] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:53.612726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:53.693530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:53.703066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:53.754993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb47467c-92de-4ea6-84ea-23926c26afa3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:53.801460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:53.802576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.803115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.803706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.810574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:53.813707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1137a-bacf-4ada-aa36-d467e991833e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:53.819677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:53.821173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b5ce471b-5484-4cd4-b168-de52df1ad3d6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:53.835883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:53.837745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 470/940] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:23:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c7b64-02b2-469e-b9d0-a7d49a472386 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.840912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c7b64-02b2-469e-b9d0-a7d49a472386 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.843694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c7b64-02b2-469e-b9d0-a7d49a472386 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.847225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 471/941] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:53 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:53.849354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c7b64-02b2-469e-b9d0-a7d49a472386 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:53.865166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:53.865779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.866048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.866476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.867026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:53.868451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:53.871440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING keystone.server.flask.common [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 24 03:23:53.897273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:23:53.897273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 03:23:53.897273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [parameters: {'id': '29a6f53a54e947eea9ef8f0159b5b930', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 03:23:53.897273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a16088b3-5d2d-4eda-9169-ec258e3b423d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 29a6f53a54e947eea9ef8f0159b5b930.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 29a6f53a54e947eea9ef8f0159b5b930. Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:23:53.905083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 24 03:23:53.906690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Aug 24 03:23:53.908643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:23:53.911816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:23:53.914317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:23:53.916216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'id': '29a6f53a54e947eea9ef8f0159b5b930', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:53.917949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:53.919343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 29a6f53a54e947eea9ef8f0159b5b930. Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 471/942] 10.222.0.91 () {60 vars in 1140 bytes} [Wed Aug 24 03:23:53 2022] POST /identity/v3/projects => generated 262 bytes in 204 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:53.921484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:53.922828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:53.941812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:53.944329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:54.003120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c7b64-02b2-469e-b9d0-a7d49a472386 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.029250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:54.054250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 472/943] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:53 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:54.054250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.063523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.064131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.064934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.065282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.066391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:54.066696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.068140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.097009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 472/944] 10.222.0.91 () {56 vars in 1149 bytes} [Wed Aug 24 03:23:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:54.097956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.184238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:54.211357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d7d878-80aa-44ee-ab55-0e6290f1d81e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.238649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548c7e8a-dec9-406d-9ca8-69a8b2f2b21f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.250394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.251369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.252578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.253150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.263291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.265632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:54.283670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:54.285648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:54.286962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:54.288008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.288431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.288833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.289195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.291107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 473/945] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:54.303435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.303864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.305242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:54.322258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:54.323884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:54.325474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:54.325474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:54.325474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:54.325989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 473/946] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:54.335029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.372755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:54.474169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c494f53-4c8b-4ba8-a074-d343fc875aef None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.496745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2749b-d204-41d7-840e-44a5b9dcbaca None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.516945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.518096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.518409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.518677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.519347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:54.519771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.522138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.547389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 474/947] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:54.548247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.550011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.550856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.551343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.551617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.552288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:54.552591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.554042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.570165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:54.580643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 474/948] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:54.581875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.713770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:54.728231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b1a6f8-d42f-4780-831a-8135c8ac7a7c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.739908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d38d5e-efcc-404d-aa0a-7efcaa1b82fc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:54.793559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.796138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.796259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:54.796408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.796408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.796580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:54.796651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.796731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:54.796731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:54.796865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.804664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:54.806115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:54.808427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:54.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:54.811492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 475/949] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:54.821108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:23:54.824051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:54.825889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:54.827099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 475/950] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:54.834452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:54.933871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:55.005816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883090a5-de84-4be6-82f6-19fe6d4cf3e7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:55.013500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c511805-5d20-4f73-9757-57137a4d0513 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:55.061740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:55.062720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.063311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.063865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.065135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:55.065606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:55.067979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:55.086156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 476/951] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:55.100278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:55.109102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:55.113755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:55.115027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.115722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.116217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.117493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:55.118035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:55.121559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:55.133517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 476/952] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:55.139009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:55.257414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d82d446-1c0f-458a-8343-33255df5c0e4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:55.284481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d861372-9c7e-4c04-adb2-bcd754f0a104 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:55.308448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:55.309464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.309964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.310440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/787f0e0c0d9d42c999c77c7eb75d4ceb/roles/d864158e9f14416d9b63a27f1796d25c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.313018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=787f0e0c0d9d42c999c77c7eb75d4ceb, role_id=d864158e9f14416d9b63a27f1796d25c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:55.314599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:55.342222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 477/953] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:23:55 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/787f0e0c0d9d42c999c77c7eb75d4ceb/roles/d864158e9f14416d9b63a27f1796d25c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:23:55.354035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:55.355000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.355269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.355538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.356186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:55.356543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:55.358082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42424d-e8d7-4994-82af-023d941c9184 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:55.362767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:55.366962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 477/954] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:55.373416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:55.479797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:55.536878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e1b24-b4ed-4aeb-a00f-410483cdc246 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.537939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e1b24-b4ed-4aeb-a00f-410483cdc246 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.538024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906e1b24-b4ed-4aeb-a00f-410483cdc246 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.540454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 478/955] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:55.547596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.547877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.548115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.567852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d3bab-c28c-4cc3-9f42-3e0563fdfe13 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:55.574228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:55.575478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:55.622923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:55.623674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:55.624337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:55.624691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:55.625546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:55.628108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:55.669971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 478/956] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:23:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:55.692367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 479/957] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:56.017949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:56.186896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9066602-70b3-44b5-bef3-e8aad64c8925 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:56.219391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:56.250736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:56.251857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:56.252338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:56.252798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:56.253659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:56.256395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:56.271545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:56.274333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:56.276474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 479/958] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:23:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:56.282841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:56.420956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d0e24b-a997-4f2f-93b4-13215c120840 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:56.448976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:56.463946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:56.465002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:56.465475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:56.465844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:56.466895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:56.467445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:56.470133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f672b00-c7e1-4a0e-832e-40561ea2af74 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:56.481121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 480/959] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:56.491359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:56.635225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f40def1-1aee-47f4-82fc-9284148b7455 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:56.674697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:56.675323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:56.675609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:56.675890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:56.676646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:56.676906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:56.678245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5bee3-c96c-4603-816f-3b8bc874ece7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:56.683897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 480/960] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:23:56 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:57.214330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea6216-4c34-4331-83c6-183c457d2997 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:57.215230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea6216-4c34-4331-83c6-183c457d2997 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:57.215474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ea6216-4c34-4331-83c6-183c457d2997 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:57.218252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 481/961] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:57.227186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0e9c-642d-4930-96d9-0faada916d4b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:57.228159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0e9c-642d-4930-96d9-0faada916d4b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:57.228159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04b0e9c-642d-4930-96d9-0faada916d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:57.254636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c04b0e9c-642d-4930-96d9-0faada916d4b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:57.256681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c04b0e9c-642d-4930-96d9-0faada916d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:57.336956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:57.380145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 481/962] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:57.873175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea6216-4c34-4331-83c6-183c457d2997 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:57.923082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfd569-a472-4baa-9d60-5ac0b3511cfc None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:57.923082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfd569-a472-4baa-9d60-5ac0b3511cfc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:57.923082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfd569-a472-4baa-9d60-5ac0b3511cfc None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:57.923082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 482/963] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:23:57.931823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:57.932799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:57.932799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:57.966179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:57.968156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:58.079604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ea6216-4c34-4331-83c6-183c457d2997 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:58.118372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 483/964] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:58.165357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:58.166459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:58.166937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:58.167417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:58.168617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:58.171410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:58.197349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.sql.core [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:23:58.197349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 03:23:58.197349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [parameters: {'user_id': 'c2abec9e92ef418893aecd17566a8025', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 03:23:58.197349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81521) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-2f407c4b-87e5-4dce-b7d3-48b2808ca981 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Aug 24 03:23:58.212074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:23:58.214135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application yield resource Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application next(self.gen) Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.rollback() Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.force_reraise() Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise self.value Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application session.commit() Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 24 03:23:58.217551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._prepare_impl() Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.session.flush() Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._flush(objects) Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application compat.raise_( Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application flush_context.execute() Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rec.execute(self) Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 24 03:23:58.223288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 24 03:23:58.229441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result = self._query(query) Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application conn.query(q) Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application result.read() Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 24 03:23:58.231585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c2abec9e92ef418893aecd17566a8025', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:58.233589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 24 03:23:58.242086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:58.243755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 482/965] 10.222.0.91 () {60 vars in 1131 bytes} [Wed Aug 24 03:23:57 2022] POST /identity/v3/users => generated 160 bytes in 343 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:58.244503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:58.244934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e4cd-90a2-4bc1-8457-6d434ec7ccb3 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:58.244934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e4cd-90a2-4bc1-8457-6d434ec7ccb3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:58.244934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5058e4cd-90a2-4bc1-8457-6d434ec7ccb3 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:58.258263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 483/966] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:58 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:58.262270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:58.262567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:58.262945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:58.292018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:58.296694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:58.374664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:58.426245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 484/967] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:23:58.463344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a05549-4c83-4074-b7c3-8586c02e9761 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:58.481493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:58.535116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:58.536593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:58.537315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:58.538380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:58.551951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:58.554900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:58.568913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:58.571835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:58.573664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:58.575510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 484/968] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:23:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:58.582663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:58.748443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd43bcdd-485b-45a6-8cc1-72118f9e8a20 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:58.785252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14ca397-5dea-479c-8ccf-29a0858ad57a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:58.798296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:58.800046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:58.800344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:58.800694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:58.823029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:23:58.831805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:58.833537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:23:58.835917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:58.837310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:58.837310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:58.837596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 485/969] 10.222.0.91 () {56 vars in 1120 bytes} [Wed Aug 24 03:23:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:23:58.837596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:58.838977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:58.839529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:58.840578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:58.841013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:58.842713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:58.894993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 485/970] 10.222.0.91 () {56 vars in 1101 bytes} [Wed Aug 24 03:23:58 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:58.895690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:58.915475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:58.936065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.029915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9160f5d7-0aa3-40e4-9f2a-7f5385ff18e0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.075189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff21d23-3f81-47b4-b47f-fa27e48e6d5e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.091405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.092310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.092735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.093146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.093977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.096274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:59.108133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:59.110092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:59.111841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 486/971] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:23:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:59.116679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.153900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.154662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.154943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.155287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.155944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:59.156303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.157953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a7827b-b584-42d2-b7a6-86d00472ee7d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.170523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 486/972] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:23:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:59.172270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db95254-b835-442e-b4da-a5baf21e6ab0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.172700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db95254-b835-442e-b4da-a5baf21e6ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.173102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db95254-b835-442e-b4da-a5baf21e6ab0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.180519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 487/973] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:23:59.180519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db95254-b835-442e-b4da-a5baf21e6ab0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.283918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad2f839-c746-4549-9ef6-dbb4c6e37c94 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.287521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.307147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.325531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db95254-b835-442e-b4da-a5baf21e6ab0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.329941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.331307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.331582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.331847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.332475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:59.332765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.334136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e0423b-9796-4f57-817a-99fc1ce6a2f1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.339780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 487/974] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:59.341517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.341926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.342250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.362094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:23:59.363330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:23:59.379641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.380622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.381100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.381483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.396402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.397863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:23:59.413518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:59.414968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:23:59.416457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:23:59.416457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:23:59.416457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:23:59.416457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 488/975] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:59.416457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.441604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 488/976] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:23:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:23:59.443517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.545540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef5154-5996-448b-8fc1-84a440856428 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.571771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.576926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c510ad-255e-42bb-9ca2-d7acdfd5f29b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.583969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.584580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.584864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.585143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.585780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:59.586086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.587504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.592856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 489/977] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:59.599329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.602459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.645260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.646337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.646867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.647453 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.649261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:59.649261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.651501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.675974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 489/978] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:23:59.681948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.736266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3abf1ea-e6d7-40a6-b11e-637f90874b31 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.781369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.781976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.782288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.782571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.783845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] RBAC: Adding query filter params (interface=admin, service_id=2900434579b44c71855033084d25a71f, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:23:59.784168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.785558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.791354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 490/979] 10.222.0.91 () {56 vars in 1249 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/endpoints?service_id=2900434579b44c71855033084d25a71f&interface=admin®ion_id=RegionOne => generated 193 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:23:59.792127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.816665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ade7208-2bae-4c1d-b3d6-c580da8743fa None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.859616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.860702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.861187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.861531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.871332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.872775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.877905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:23:59.884731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:23:59.886593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 490/980] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:23:59.888084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:23:59.910894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:23:59.934818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93ab334-49a3-4e1a-b67d-e79205b40cae None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:23:59.977035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:23:59.977723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:23:59.978062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:23:59.978444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:23:59.979161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:23:59.980615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:23:59.996903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 491/981] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:23:59 2022] POST /identity/v3/services => generated 242 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:23:59.997601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:00.024268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293b6d72-13b7-4c7c-8275-c03b37dab3dc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:00.094352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:00.095309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:00.096333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:00.096814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:00.097850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:00.100230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:00.115275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 491/982] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/services => generated 1727 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:24:00.179673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa54c05-d346-4871-ba56-1343e6e22c25 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:00.231889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:00.248122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:00.248951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:00.249481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:00.249821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:00.250720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:00.251044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:00.253321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:00.277845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 492/983] 10.222.0.91 () {56 vars in 1149 bytes} [Wed Aug 24 03:23:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:00.284093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:00.405381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:00.433837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338b10b5-020f-4eea-a0b6-c161b6e69272 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:00.477221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:00.478630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:00.479152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:00.479583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:00.561581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:00.564525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:00.664682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:00.692756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 492/984] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:00 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:24:00.699363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:00.817510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2435a066-1755-43b3-9b6b-7c8a09cf0d87 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:00.857161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:00.858366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:00.858942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:00.859280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:00.870808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:00.872330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:00.888598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:00.889755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:00.891159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:00.891159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:00.891159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:00.894726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 493/985] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:00.896715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:01.028205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9ecaa7-8e08-4590-ad39-8074d4c8010c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:01.083687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:01.084612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:01.085083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:01.085540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:01.086487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:01.086994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:01.088533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:01.122264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 493/986] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:01.124639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:01.166567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:01.245869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364439b-a06f-465d-8a57-98ce165a741a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:01.284514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:01.285260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:01.285671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:01.286086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:01.294095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:01.295799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:24:01.308061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:01.309230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:01.314076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 494/987] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:01.315774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:01.367265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:01.439211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96834c87-a8cb-4606-83cf-2cc5e962f433 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:01.475570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:01.476177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:01.476456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:01.476754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:01.477413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:01.477704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:01.479157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73923813-bc0c-4254-a709-9b2bc383836f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:01.497587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 494/988] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:01.501713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:01.635422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:01.643399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba05ad24-71d2-47c0-859b-1c48006a9502 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:01.685531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:01.686136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:01.686445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:01.686712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:01.687560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] RBAC: Adding query filter params (role.id=d864158e9f14416d9b63a27f1796d25c, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=787f0e0c0d9d42c999c77c7eb75d4ceb) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:01.687869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:01.689313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cb2f76-1c77-43ab-a792-112c07765107 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:01.696367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 495/989] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:01 2022] GET /identity/v3/role_assignments?user.id=787f0e0c0d9d42c999c77c7eb75d4ceb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=d864158e9f14416d9b63a27f1796d25c => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:02.065666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09946-ba58-4649-ab1c-ec20a3756392 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:02.066409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09946-ba58-4649-ab1c-ec20a3756392 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:02.066906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d09946-ba58-4649-ab1c-ec20a3756392 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:02.068736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 495/990] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:02.073860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:02.074360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:02.074557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:02.095232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:02.097114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:02.178360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 496/991] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:02.448769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09946-ba58-4649-ab1c-ec20a3756392 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:02.554648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:02.560268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d09946-ba58-4649-ab1c-ec20a3756392 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:02.605334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:02.606349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:02.606826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:02.607331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:02.608328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:02.610746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e330873b-c3d2-44c3-ac38-7733700216f4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:02.625567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:02.626693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:24:02.627583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:02.630384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 496/992] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:24:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:02.633752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:02.740875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:02.752727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8b7a6a-9ab1-4b97-be4f-a40b25523aa5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:02.799519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:02.800240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:02.800520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:02.800784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:02.801414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:02.801710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:02.803110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237e91e1-d753-4d44-89e2-e647eb34a3d4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:02.813761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7db43-ef4e-4726-8634-0441ee2df9ea None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:02.814158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7db43-ef4e-4726-8634-0441ee2df9ea None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:02.814605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e7db43-ef4e-4726-8634-0441ee2df9ea None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:02.816044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 497/993] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:02.816763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 497/994] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:24:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:02.822127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7db43-ef4e-4726-8634-0441ee2df9ea None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:02.825421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:02.825777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:02.827227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:02.855530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:02.856764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:02.982226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 498/995] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:02 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:02.984692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e7db43-ef4e-4726-8634-0441ee2df9ea None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:03.026661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:03.027554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.027986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.028404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.029197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:03.029645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:03.031326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:03.037596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 498/996] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:24:02 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:03.045146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:03.179608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:03.202571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d4a5a-27f5-419a-b0ef-0c1286e68a15 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:03.241857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:03.242565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.242846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.243112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.243996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] RBAC: Adding query filter params (interface=public, service_id=fa05f2536ac947e5906048c85b400dcf, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:03.244301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:03.245677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:03.251498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 499/997] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:24:03 2022] GET /identity/v3/endpoints?service_id=fa05f2536ac947e5906048c85b400dcf&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:03.258264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:03.262756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:03.312609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:03.398940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ea02f-d674-4e65-a3ea-bc815d647e14 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:03.432383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83732-91e4-4a26-a764-155c4fdaecd9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:03.446382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:03.447382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.447851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.448322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.449308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:03.451589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:03.462335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:03.463383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:03.464258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 500/998] 10.222.0.91 () {56 vars in 1126 bytes} [Wed Aug 24 03:24:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:03.475032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:03.489339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:03.490111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.490268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.490515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.491129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:03.492494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:03.496118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:03.500574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 499/999] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:24:03 2022] GET /identity/v3/services => generated 1727 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:24:03.632973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9884-9a30-4910-868e-1022589068a5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:03.680625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:03.681239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.681518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.681790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.682459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:03.682764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:03.684152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228bd05d-594b-4bcc-877b-b1773a91c4a4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:03.693108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 501/1000] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:24:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:03.702456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:03.864864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:03.878869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737eeda5-5215-49ea-b27f-93046fa1965c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:03.916641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:03.917263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.917544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.917817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.918487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:03.918787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:03.920275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:03.932707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 500/1001] 10.222.0.91 () {56 vars in 1122 bytes} [Wed Aug 24 03:24:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:03.932938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66120e3-7c25-44b7-be92-17ea57a6ed86 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.932938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66120e3-7c25-44b7-be92-17ea57a6ed86 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.932938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66120e3-7c25-44b7-be92-17ea57a6ed86 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.934838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 502/1002] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:03.936218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:03.940829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:03.941180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:03.941462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:03.960208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:03.961421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:04.046644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 503/1003] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:03 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:04.120786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4ba33d-b401-4377-8fc4-e7b5a09e0242 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:04.177812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:04.178802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:04.179291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:04.179786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:04.180994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:04.183632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:04.214319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 501/1004] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:24:03 2022] POST /identity/v3/endpoints => generated 325 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:24:04.340324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:04.350089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:04.508781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea44307c-aa8d-4137-8fcc-7d1d002dcea6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:04.561444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:04.562084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:04.562433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:04.562709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] PATH_INFO: `/v3/services/image` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:04.563375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:04.565220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:04.578052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:04.579928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:04.581518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 504/1005] 10.222.0.91 () {56 vars in 1105 bytes} [Wed Aug 24 03:24:04 2022] GET /identity/v3/services/image => generated 86 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:04.588246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:04.617137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:04.738409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab0fe79-1f40-4a55-9eb7-095729639582 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:04.786966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:04.786966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:04.789305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:04.789305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:04.789305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] RBAC: Adding query filter params (name=image) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:04.789305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:04.792022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8500dc99-6905-46b3-8876-86eccae6006b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:04.805055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 502/1006] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:04 2022] GET /identity/v3/services?name=image => generated 122 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:04.806689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:04.888783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:04.951010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18eda09-4bfc-4a20-a3e5-740039b68624 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:04.998922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:04.999786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:05.000227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:05.000661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:05.001712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] RBAC: Adding query filter params (type=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:05.002040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:05.003696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64a8208-52aa-40ad-a1bd-1b73af493caa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:05.018261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 505/1007] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:04 2022] GET /identity/v3/services?type=image => generated 122 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:05.577349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ed8d-12d5-40ef-bed0-fb79dfd97ba8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:05.577696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ed8d-12d5-40ef-bed0-fb79dfd97ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:05.578120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0929ed8d-12d5-40ef-bed0-fb79dfd97ba8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:05.579471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 503/1008] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:05.586084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c9986-5510-4871-8f02-9c6bad269058 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:05.586883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c9986-5510-4871-8f02-9c6bad269058 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:05.586883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c9986-5510-4871-8f02-9c6bad269058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:05.610599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-da0c9986-5510-4871-8f02-9c6bad269058 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:05.613384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-da0c9986-5510-4871-8f02-9c6bad269058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:05.718572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 506/1009] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:05 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:06.066804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ed8d-12d5-40ef-bed0-fb79dfd97ba8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:06.191841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:06.223754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0929ed8d-12d5-40ef-bed0-fb79dfd97ba8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:06.262379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350df47-3ecc-4752-a2ac-e69d83e2722b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.262733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350df47-3ecc-4752-a2ac-e69d83e2722b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.262929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b350df47-3ecc-4752-a2ac-e69d83e2722b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.264426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 507/1010] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:06.270110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.270499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.270751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.284022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:06.285114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.285606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.286094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.287182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:06.289988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:06.290618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:06.291850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:06.303272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:06.305308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:06.307064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 504/1011] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:24:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:06.310577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:06.384093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:06.406377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 508/1012] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:06 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:06.444398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec15001d-473f-4974-8ddc-637c2e3f633f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:06.490773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:06.491783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.492260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.492737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.493857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:06.494427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:06.496719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:06.510314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 505/1013] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:24:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:06.511506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:06.630788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d16ce9-4cf4-4488-a3c1-f1a3a3fbcceb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:06.669293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:06.669905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.670228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.670485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.671126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:06.671436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:06.672833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:06.683376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 509/1014] 10.222.0.91 () {56 vars in 1112 bytes} [Wed Aug 24 03:24:06 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:06.685317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:06.697117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:06.810314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3837c5d6-aab9-4173-8dc5-155d5f249d7c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:06.857275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:06.858272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:06.858705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:06.859174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:06.860251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:06.862830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:06.894219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 506/1015] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:24:06 2022] POST /identity/v3/endpoints => generated 334 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:06.901253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:07.016651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:07.097068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b88740-3a48-41c2-aa04-4834ede72d3d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:07.175585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8b29f5-7108-4fc6-b48b-5c833229c159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:07.176230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b29f5-7108-4fc6-b48b-5c833229c159 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:07.176507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b29f5-7108-4fc6-b48b-5c833229c159 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:07.176775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8b29f5-7108-4fc6-b48b-5c833229c159 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:07.177932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b29f5-7108-4fc6-b48b-5c833229c159 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:07.179544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8b29f5-7108-4fc6-b48b-5c833229c159 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:07.181430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 510/1016] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:07.187592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:07.284886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:07.352301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50348e3c-5c36-4109-ae3d-11c2bd870cb3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:07.396729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:07.397529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:07.397932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:07.398369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:07.399164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:07.400765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:07.433047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1aee-d9e6-44b9-9925-e6838fc92e01 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:07.433275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1aee-d9e6-44b9-9925-e6838fc92e01 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:07.433638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b1aee-d9e6-44b9-9925-e6838fc92e01 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:07.435117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 511/1017] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:07.438344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 507/1018] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:24:07 2022] POST /identity/v3/projects => generated 311 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:07.444488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a8aac-3fe2-4489-bbb6-eedfe1f4a844 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:07.445714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a8aac-3fe2-4489-bbb6-eedfe1f4a844 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:07.446369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04a8aac-3fe2-4489-bbb6-eedfe1f4a844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:07.472306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d04a8aac-3fe2-4489-bbb6-eedfe1f4a844 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:07.473710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d04a8aac-3fe2-4489-bbb6-eedfe1f4a844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:07.575704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 512/1019] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:07 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:07.869300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:08.007546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8d327-1a12-4027-b322-07d9d5aa72ee None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:08.025284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:08.060727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cc140-5343-4848-86b1-cf96f190b1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:08.062110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cc140-5343-4848-86b1-cf96f190b1ab None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:08.062834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cc140-5343-4848-86b1-cf96f190b1ab None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:08.063515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cc140-5343-4848-86b1-cf96f190b1ab None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:08.065459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cc140-5343-4848-86b1-cf96f190b1ab None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:08.068332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cc140-5343-4848-86b1-cf96f190b1ab None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:08.097644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 508/1020] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:24:07 2022] POST /identity/v3/services => generated 242 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:09.123305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf203bd-3805-4306-8a57-1c00c6f26e4d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.124514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf203bd-3805-4306-8a57-1c00c6f26e4d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.124887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf203bd-3805-4306-8a57-1c00c6f26e4d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.126821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 513/1021] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:09.133452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db46a-5d17-4e26-ba50-5b3f31083880 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.135131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db46a-5d17-4e26-ba50-5b3f31083880 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.135131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7db46a-5d17-4e26-ba50-5b3f31083880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.162810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8a7db46a-5d17-4e26-ba50-5b3f31083880 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:09.164424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8a7db46a-5d17-4e26-ba50-5b3f31083880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:09.270255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 509/1022] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:09.611791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf203bd-3805-4306-8a57-1c00c6f26e4d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:09.631541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78cd50-4f1f-4317-837e-52293fe10bd3 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.632010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78cd50-4f1f-4317-837e-52293fe10bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.632483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78cd50-4f1f-4317-837e-52293fe10bd3 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.634276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 510/1023] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:09.640766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.641245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.641726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.670052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:09.671945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:09.696091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:09.775889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 511/1024] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:09 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:09.776157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf203bd-3805-4306-8a57-1c00c6f26e4d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:09.818635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:09.819276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:09.819562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:09.819833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:09.833889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:09.841688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:09.846684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:09.847982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 514/1025] 10.222.0.91 () {56 vars in 1120 bytes} [Wed Aug 24 03:24:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:09.851789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:09.895588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:09.982593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9886db-f3ec-45f1-b06a-9f7bfa86a238 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.028873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.029485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.029767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.030041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.030749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:10.031049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.032518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.050214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 512/1026] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:24:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:10.054511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.089645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.222365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:10.225200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad8c4c-b8e6-4f01-ade6-be3355bc8f63 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.233520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f2fcc3-2813-43d2-8f0d-644da6f36db9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.235353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:10.267672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.269243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.269775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.270610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.277369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb07a2d-298c-4d48-a4cb-ffb8e46c0b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.277995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb07a2d-298c-4d48-a4cb-ffb8e46c0b9a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.278388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb07a2d-298c-4d48-a4cb-ffb8e46c0b9a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.278900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb07a2d-298c-4d48-a4cb-ffb8e46c0b9a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.280759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb07a2d-298c-4d48-a4cb-ffb8e46c0b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.281662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.283213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.283427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb07a2d-298c-4d48-a4cb-ffb8e46c0b9a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.286818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 513/1027] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:10.288287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219b40-acec-4aa3-aed1-9f4c9fa0e482 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.288856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219b40-acec-4aa3-aed1-9f4c9fa0e482 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.289176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16219b40-acec-4aa3-aed1-9f4c9fa0e482 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.294283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 514/1028] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:10.294283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219b40-acec-4aa3-aed1-9f4c9fa0e482 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b83c45b3-63ac-420c-aad9-5ee4c87588b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:10.306156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:10.307735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:10.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:10.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:10.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:10.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 515/1029] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:10.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.309140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.327885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:10.329084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:10.404056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 516/1030] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:10 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:10.404908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.440729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16219b40-acec-4aa3-aed1-9f4c9fa0e482 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.494103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.494674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.495096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.495504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.496160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.497740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.523327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 515/1031] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:24:10 2022] POST /identity/v3/users => generated 285 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:24:10.532058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:10.546539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f8ea-f4ec-494a-83ce-49a5712f2aa7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.590303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.590933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.591231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.591506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.592133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:10.592431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.593843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.609317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 517/1032] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:10.616133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.640975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:10.679118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.775408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0bd80-4dbb-4499-9cdf-79d77302dfd5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.822173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13f0ace-5ee5-4a8a-8b5d-458bf43c0ab8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:10.825707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.826594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.826881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.827152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.836133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.837589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:24:10.852047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:10.853213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:10.854245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 516/1033] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:10.865302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.873067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:10.873703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:10.873979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:10.874280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] PATH_INFO: `/v3/services/image` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:10.875321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:10.877088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:10.886897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:10.888231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:10.890315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:10.893774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 518/1034] 10.222.0.91 () {56 vars in 1105 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/services/image => generated 86 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:10.894764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:10.916018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:10.968758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.008617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da839b1-e8fa-42a1-8d62-642f28f2863f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.041097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbada58-0a2d-4600-8893-b413c68d3f12 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.052521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.053140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.053415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.053686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.054549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:11.054851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.056283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.087875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.088618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.088759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.089041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.089678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] RBAC: Adding query filter params (name=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:11.089984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.091422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.097483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 519/1035] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/services?name=image => generated 122 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:11.103096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.113687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 517/1036] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:24:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:11.114829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.191750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.220538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83d8070-7c22-4820-baed-39227955c846 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.257296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578634ea-e402-4642-b1b3-d078d643244e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.264209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.265170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.265621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.266047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.267115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] RBAC: Adding query filter params (type=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:11.267678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.270674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.279028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 520/1037] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:11.290449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.315597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.316117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.316388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.316649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.319664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.402812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.404254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.443050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14eeb96-9dad-441a-a68f-5eb3e470a863 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.485273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.485891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.486163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.486458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.487195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] RBAC: Adding query filter params (interface=public, service_id=03f686dc33c946be85c56fb0bb21ecc1, region_id=RegionOne) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:11.487551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.488951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.499966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 521/1038] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/endpoints?service_id=03f686dc33c946be85c56fb0bb21ecc1&interface=public®ion_id=RegionOne => generated 194 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:11.501307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.516259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.550328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 518/1039] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:24:11.553385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.620887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.639632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6dd15c-c2dc-4f46-9919-a8ad7a06a8e8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.682639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.683255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.683530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.683800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.684507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.685834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.699419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 522/1040] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/services => generated 1957 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:24:11.730656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278a729-867c-4d55-8818-d4aa13dbbe68 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:11.782813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:11.783466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:11.783778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:11.784051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:11.796762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:11.798269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:11.815867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:11.816949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:11.818421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:11.818421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:11.818421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:11.818421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 519/1041] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:11.823041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:11.921926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:11.964531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207dcf05-9807-4f4c-9a27-421d6e41f1ba None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:12.006814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:12.007777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.008178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.008582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.009371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:12.009799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:12.011362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:12.025579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 523/1042] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:12.031830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:12.145329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:12.165360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b774d-b016-4434-9c0a-a2012ef2232e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:12.206313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:12.206868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.207143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.207411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.214366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:12.215898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:24:12.228899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:12.229934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:12.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 520/1043] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:24:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:12.236016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:12.362674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef54f174-d832-4705-895c-4304e876cd49 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:12.399947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:12.407252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:12.407901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.408175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.408450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.409723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:12.410028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:12.411530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba681295-43d2-4043-8b0e-d71c53bbbd87 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:12.424685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 524/1044] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:24:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:12.430962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:12.557619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390a1c61-6d0e-4814-bd68-0fe01cb4dfa6 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:12.583559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc5bb6-3542-4837-8d85-a70690d8f273 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.583928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc5bb6-3542-4837-8d85-a70690d8f273 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.584135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc5bb6-3542-4837-8d85-a70690d8f273 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.585444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 525/1045] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:12.591248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.591524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.591796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.603810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:12.604431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:12.604705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:12.604973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:12.605699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] RBAC: Adding query filter params (role.id=d864158e9f14416d9b63a27f1796d25c, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=1d56ed96c2f546e59e473b1d62bf99eb) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:12.606003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:12.607621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:12.618230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 521/1046] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:12 2022] GET /identity/v3/role_assignments?user.id=1d56ed96c2f546e59e473b1d62bf99eb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=d864158e9f14416d9b63a27f1796d25c => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:12.619477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:12.620585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:12.754890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 526/1047] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:13.028508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:13.050251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:13.173253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1d42cd-a7cb-4814-a36e-722ab7174fb5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:13.231656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:13.232299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.232576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.232854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.247368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:13.252274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:13.254645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:13.256253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 522/1048] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:24:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:13.260712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:13.407777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9eaf1-4cce-40c3-8e76-edced30a6d58 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:13.442641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:13.466175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:13.466986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.467392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.467790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.468600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:13.469042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:13.470626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:13.481116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 527/1049] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:24:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:13.486660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:13.576212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:13.624946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3179df-b0b3-4c32-924d-59619c969cad None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:13.663074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac492497-7bea-4726-98a8-d1f5ac3909a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:13.663600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac492497-7bea-4726-98a8-d1f5ac3909a6 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.664160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac492497-7bea-4726-98a8-d1f5ac3909a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.664411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac492497-7bea-4726-98a8-d1f5ac3909a6 None admin] PATH_INFO: `/v3/users/9ac5d8e3d1d04c0a8fb565ab39a1c351` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.686052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac492497-7bea-4726-98a8-d1f5ac3909a6 None admin] RBAC: Authorizing `identity:get_user(user_id=9ac5d8e3d1d04c0a8fb565ab39a1c351)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:13.687547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac492497-7bea-4726-98a8-d1f5ac3909a6 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:13.689555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 523/1050] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:13 2022] GET /identity/v3/users/9ac5d8e3d1d04c0a8fb565ab39a1c351 => generated 285 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:13.697193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:13.824868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92cd8e2-4e07-4b39-85e6-33d3d01632c4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:13.866479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:13.866988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.867199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.867434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.874534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9cb4c-e937-4873-a9ba-24aafe2fa336 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.874847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9cb4c-e937-4873-a9ba-24aafe2fa336 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.875077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be9cb4c-e937-4873-a9ba-24aafe2fa336 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.876660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 524/1051] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:13.882178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:13.882559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:13.882844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:13.900969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:13.902175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:13.934987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:13.965701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:13.967386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:13.999798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 525/1052] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:14.114374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 528/1053] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:13 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:24:14.117402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:14.185288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:14.242284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a8754c-f15f-456b-a6d0-035e4c71a68e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:14.289447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:14.290090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.290442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.290702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.291845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] RBAC: Authorizing `identity:get_project(project_id=373d1e6c3f8041e6be98b77d917f3161)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:14.293306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:14.295155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 526/1054] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:14 2022] GET /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161 => generated 311 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:14.301915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:14.303952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:14.411717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:14.430621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae7c046-b064-4f60-aeb5-c49bd3bf7f53 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:14.437580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43319990-fd1c-4b28-917d-e352c2522280 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:14.464106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:14.482036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:14.483028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.483373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.483648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] PATH_INFO: `/v3/services/image` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.483988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:14.484233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:14.484590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.484934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.485209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.485590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:14.485941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=373d1e6c3f8041e6be98b77d917f3161, user.id=9ac5d8e3d1d04c0a8fb565ab39a1c351) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:14.486296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:14.488373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-c6a6177f-0617-4755-ace9-a4cea01715aa None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:14.495682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:14.497377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 24 03:24:14.498431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:14.499373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 529/1055] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:14 2022] GET /identity/v3/role_assignments?user.id=9ac5d8e3d1d04c0a8fb565ab39a1c351&scope.project.id=373d1e6c3f8041e6be98b77d917f3161&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:14.506210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 527/1056] 10.222.0.91 () {56 vars in 1105 bytes} [Wed Aug 24 03:24:14 2022] GET /identity/v3/services/image => generated 86 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:14.508047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:14.642755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd451e-afc4-4f48-912b-b93754b44b76 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.643881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd451e-afc4-4f48-912b-b93754b44b76 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.646637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcd451e-afc4-4f48-912b-b93754b44b76 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.650575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 528/1057] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:14 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:14.656133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.656410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.656681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.663523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a42d8-d279-4260-bb72-030714bca457 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:14.678571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:14.680108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:14.710232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:14.711129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.711582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.712124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.712274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:14.712816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:14.713956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:14.725633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 530/1058] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:14.732034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:14.748035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:14.772079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 529/1059] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:14.861256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db2d36-ea26-43f0-af1e-70a922b6a9f2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:14.903878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:14.904483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:14.904771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:14.905023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:14.905640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] RBAC: Adding query filter params (type=image) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:14.905939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:14.907345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:14.920247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 531/1060] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:14.921389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:15.050924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:15.067741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a6a986-7128-4b6d-92fe-063b48e51115 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:15.101894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:15.125769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:15.126672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:15.126955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:15.127238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:15.127922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:15.129346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:15.145624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:15.161752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 530/1061] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:24:14 2022] POST /identity/v3/endpoints => generated 327 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:15.184680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd1613a-ba9a-44e2-bdd6-4bed72cb0176 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:15.243016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:15.243666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:15.243946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:15.244218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:15.257480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:15.262674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:15.263841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:15.264846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 532/1062] 10.222.0.91 () {56 vars in 1120 bytes} [Wed Aug 24 03:24:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:15.273705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:15.396932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875a59f5-1a6e-492c-a881-e5663e8bfb9e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:15.445011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:15.445976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:15.446457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:15.446916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:15.447793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:15.448309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:15.450021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:15.462664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 531/1063] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:24:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:15.472131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:15.599708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28188a-8f30-40a2-ac83-befc5cf2050c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:15.633615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:15.641898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:15.642706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:15.642982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:15.643754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:15.658081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:15.659776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:15.682497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:15.684361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:15.685490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:15.685490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:15.685490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:15.685490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 533/1064] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:15.692567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:15.747123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:15.827709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554cc2b9-6932-487d-a495-3288a42c96e8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:15.867631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:15.868418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:15.868839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:15.869249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:15.870041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:15.870513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:15.872098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:15.885592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 532/1065] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:15.892697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.003354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b392d-b8fc-40f3-8b52-2a46cd507f73 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:16.040753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:16.041385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.041658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.041926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] PATH_INFO: `/v3/users/nova` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.050231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:16.051682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:24:16.063373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:16.064476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:24:16.065588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:16.069483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 534/1066] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:24:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:16.070793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.185457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:16.192572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa42f3-513d-4b81-b75d-df9ee6218dce None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:16.233635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:16.234379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.234660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.234927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.235686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:16.235988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:16.238018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657470c6-6529-4c9b-97da-b78f58cf7efa None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:16.259551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 533/1067] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:16.264668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.319788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:16.385668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c5ca9-286d-4fd6-ba8d-f3d2a71984be None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:16.426983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:16.428831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.428831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.428831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.441093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:16.442712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:16.457135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:16.464970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:16.467033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:16.467033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:16.467033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:16.467033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 535/1068] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:16.468439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b2493-786b-4473-a79f-3b3a0ac71e70 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.468722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b2493-786b-4473-a79f-3b3a0ac71e70 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.468986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5b2493-786b-4473-a79f-3b3a0ac71e70 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.470393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 534/1069] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:16.470998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.475911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.476190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.476460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.495184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:16.496364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:16.596151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:16.602665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 535/1070] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:16.604880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9ab41-23ef-4a05-882a-43b15180a53e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:16.645862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:16.646805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.647331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.647895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.648724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:16.649204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:16.650781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:16.665007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 536/1071] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:16.671138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.801210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:16.823669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2e225b-9bf3-4681-a7b9-b733e32a1337 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:16.859695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.874193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:16.874838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:16.875111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:16.875378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:16.882404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:16.883834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:24:16.896181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:16.897137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:16.898045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 536/1072] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:16.903298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:16.998393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb76ff0a-63aa-4414-b23d-73eaf4ff93a3 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.019192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44acc5af-ae03-401f-9b31-70223726c773 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.055872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.056484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.056752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.057021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.057325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.057707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:17.058005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.059208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.059461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.060159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.060656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.074078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:17.076941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 537/1073] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:17.079792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:17.081428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:17.082301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:17.086120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:17.086464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 537/1074] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:24:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:17.087344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:17.099149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.114848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.220241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b744c26d-2217-42a2-8888-183e069ea43e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.228385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3196e5-19a2-4aaa-a652-109e62277438 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.262403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.262931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.263202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.263466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.264126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:17.264436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.265817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6471e301-9ac1-4908-b949-5d3dbb3be085 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.269139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.269806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.270097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.270393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/1d56ed96c2f546e59e473b1d62bf99eb/roles/d864158e9f14416d9b63a27f1796d25c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.272531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=1d56ed96c2f546e59e473b1d62bf99eb, role_id=d864158e9f14416d9b63a27f1796d25c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.273954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.275682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 538/1075] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:24:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:17.276937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4466e-b0ca-457c-b605-3f5e6ca8a4f6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.277223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4466e-b0ca-457c-b605-3f5e6ca8a4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.277473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb4466e-b0ca-457c-b605-3f5e6ca8a4f6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.278990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 539/1076] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:17.284531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.284820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.285203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.294864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 538/1077] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:24:17 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/1d56ed96c2f546e59e473b1d62bf99eb/roles/d864158e9f14416d9b63a27f1796d25c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:24:17.295704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:17.306686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:17.309073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:17.378737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.413795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 540/1078] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:17.423687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7653cece-0d70-467a-8152-0d6610e9f50e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.464661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.465357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.465635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.465910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] PATH_INFO: `/v3/users/9ac5d8e3d1d04c0a8fb565ab39a1c351` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.467086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] RBAC: Authorizing `identity:get_user(user_id=9ac5d8e3d1d04c0a8fb565ab39a1c351)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.468514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.470120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 539/1079] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:17 2022] GET /identity/v3/users/9ac5d8e3d1d04c0a8fb565ab39a1c351 => generated 285 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:17.476276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:17.490052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.643151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3dd6e-7c08-44ab-9ca4-6dcde7692552 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.683655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.684287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.684568 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.684842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.686051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] RBAC: Authorizing `identity:get_project(project_id=373d1e6c3f8041e6be98b77d917f3161)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.687524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.691214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 541/1080] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:17 2022] GET /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161 => generated 311 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:17.697540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:17.721746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:17.817188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.845584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d3464-52a6-4430-b774-d3b0a70f35d9 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.849098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3bb31-92cc-4d0a-a5cc-a1ca0c631868 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:17.889923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.890642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.890919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.891396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161/users/9ac5d8e3d1d04c0a8fb565ab39a1c351/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.893473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] RBAC: Authorizing `identity:create_grant(project_id=373d1e6c3f8041e6be98b77d917f3161, user_id=9ac5d8e3d1d04c0a8fb565ab39a1c351, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.894984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:17.898921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:17.901773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:17.902771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:17.903342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:17.903788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:17.911347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 540/1081] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:24:17 2022] PUT /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161/users/9ac5d8e3d1d04c0a8fb565ab39a1c351/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:24:17.995378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:17.996910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:18.108248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:18.143264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 542/1082] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:17 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:24:18.151757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:18.280827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6ca014-436f-4116-b88f-40e5c2123649 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:18.326468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:18.327299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:18.327576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:18.327842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:18.349476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:18.350953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1519e48d-2fb4-4356-91c1-69d26e06bf0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:18.367676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:18.368738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:18.375358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:18.375358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:18.375358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:18.375358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 541/1083] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:18.377107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:18.524077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e1987d-8592-465d-92fd-304924470ca9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:18.565687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:18.566310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:18.566586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:18.566954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:18.567605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:18.567899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:18.569267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8cd38-1f97-408c-bb24-67b1efd32e06 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:18.591828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 543/1084] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:19.262068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6752a2-80a2-46bf-9e9e-51899e6865ac None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.262643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6752a2-80a2-46bf-9e9e-51899e6865ac None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.262643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6752a2-80a2-46bf-9e9e-51899e6865ac None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.263990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 542/1085] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:19.270919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.271401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.271498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.289761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:19.290977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:19.319669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:19.361435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 544/1086] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:19.636317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6752a2-80a2-46bf-9e9e-51899e6865ac None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:19.684629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:19.740806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6752a2-80a2-46bf-9e9e-51899e6865ac None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:19.774889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae7cb7-9c47-4add-9aa6-409c1ed2da97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:19.775532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae7cb7-9c47-4add-9aa6-409c1ed2da97 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.775804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae7cb7-9c47-4add-9aa6-409c1ed2da97 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.776064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae7cb7-9c47-4add-9aa6-409c1ed2da97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.787666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae7cb7-9c47-4add-9aa6-409c1ed2da97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f9ae7cb7-9c47-4add-9aa6-409c1ed2da97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:19.792578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:19.793778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:19.794676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 543/1087] 10.222.0.91 () {56 vars in 1120 bytes} [Wed Aug 24 03:24:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:19.800726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:19.879172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d546101-121f-466d-b757-64f9d5a77875 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.879457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d546101-121f-466d-b757-64f9d5a77875 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.879736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d546101-121f-466d-b757-64f9d5a77875 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.881092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 544/1088] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:19.886485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.886705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.886979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.905215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:19.906452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:19.912744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c08283-edb0-4471-a5f2-5c69acc26ef9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:19.954575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:19.955182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:19.957122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:19.957396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:19.958067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:19.958386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:19.959763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:19.968335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 545/1089] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:24:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:19.974602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:19.984217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 545/1090] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:20.001548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.076792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c82813-1a87-4432-bb87-3939fea6785b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.111324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.111954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.112314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.112581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.123077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.124512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:20.138278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:20.139348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:20.140268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:20.140268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:20.140268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:20.140268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 546/1091] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:20.145709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.181186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.249685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f17695-4ca3-4cb3-9802-a0a12f3a23db None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.255845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.286909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.287502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.287794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.288060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.288706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:20.289007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.290670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.302537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 546/1092] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:20.308211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.336778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.358280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c2a79a-6cc4-4f98-a4c1-8596bf464943 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.393352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.394543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.394836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.395083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.405993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:20.410618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:20.411157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:20.412316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:20.413391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 547/1093] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:20.418757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.420831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff6f828-1ff9-4231-a521-e0b5b9bf5510 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.454038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.454695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.454974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.455244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] PATH_INFO: `/v3/users/nova` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.463756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.465195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:24:20.476147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:20.478028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:20.479483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 547/1094] 10.222.0.91 () {56 vars in 1093 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:20.482997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.526668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4201bd70-4031-4620-aacf-8c6468a77dd2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.560107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.560772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.561027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.561341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.562057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:20.562391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.563774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.569738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.574340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 548/1095] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:20.579911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.588219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba941d9-844c-4df0-a131-90934fbff1ae None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.624522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.625136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.625409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.625682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.626439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:20.626742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.628851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.646950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 548/1096] 10.222.0.91 () {56 vars in 1133 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:20.652470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.668748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.688169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb24ab6-f3a2-4309-9c7f-aef0ba1c0270 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.721989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.722778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.723056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.723340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] PATH_INFO: `/v3/users/9ac5d8e3d1d04c0a8fb565ab39a1c351` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.724520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] RBAC: Authorizing `identity:get_user(user_id=9ac5d8e3d1d04c0a8fb565ab39a1c351)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.725920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.727687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 549/1097] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/users/9ac5d8e3d1d04c0a8fb565ab39a1c351 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:20.733459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:20.755194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfb6f7-8f9b-423b-90a2-e94600f583ea None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.789743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.790384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.790665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.790937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.798956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.835209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad8916b-47d7-489c-9b60-55a14d3169df None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:20.860402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.861845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.869095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:20.869699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:20.869985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:20.870287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:20.877250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:20.947187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:20.948618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:20.972640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 549/1098] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:24:20.978537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.028511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.069685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 550/1099] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 24 03:24:21.075535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.081244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d999c-5e55-4270-b834-513c416a7e0e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.098966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.123391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.124057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.124312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.124585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.135222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.136536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:21.151501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:21.152586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:24:21.153508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:21.153508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:24:21.153508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:21.153508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 550/1100] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:21.158273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.186298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f62e45-585a-456c-949e-0a45a448a3d2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.221226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.222317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.222798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.222911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.223965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] RBAC: Authorizing `identity:get_project(project_id=373d1e6c3f8041e6be98b77d917f3161)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.225413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.227093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 551/1101] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:21.233156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.266517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d48870a-fe98-4ca7-9002-e996e1fcd941 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.269098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.311876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.312492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.312772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.313047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.313855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:21.314018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.315436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.328225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 551/1102] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:21.334670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.342997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d050d18-b826-4985-a409-4fd2696a6214 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.362272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.378972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.379571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.379845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.380117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.380851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=373d1e6c3f8041e6be98b77d917f3161, user.id=9ac5d8e3d1d04c0a8fb565ab39a1c351) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:21.381151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.382622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.388562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 552/1103] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/role_assignments?user.id=9ac5d8e3d1d04c0a8fb565ab39a1c351&scope.project.id=373d1e6c3f8041e6be98b77d917f3161&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:21.442357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b8664-785a-48e5-b1ed-f6b7490b22cf None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.480815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.481379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.481656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.481929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.489647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.491158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:24:21.503495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:21.504585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:24:21.505510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:21.506711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 552/1104] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:24:21.510758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.616981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.620196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720db030-c305-4f67-af09-0f8d98053d82 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.656618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.657218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.657527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.657799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.658523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:21.658825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.660255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4f0d64-c7b7-4036-b4dd-367c5c30b732 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.672755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 553/1105] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:21.682503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:21.767526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:21.782327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a4b828-05ef-46bc-bd9a-b29a9f6635d8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:21.817479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:21.818089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:21.818393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:21.818662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:21.819386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] RBAC: Adding query filter params (role.id=d864158e9f14416d9b63a27f1796d25c, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=1d56ed96c2f546e59e473b1d62bf99eb) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:21.819692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:21.821126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dad7f-b8e3-43c0-b583-db1b16d39b77 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:21.827145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 553/1106] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:21 2022] GET /identity/v3/role_assignments?user.id=1d56ed96c2f546e59e473b1d62bf99eb&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=d864158e9f14416d9b63a27f1796d25c => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:23.350287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a542f2-6b8f-46b6-acfc-894e62b179f0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:23.350826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a542f2-6b8f-46b6-acfc-894e62b179f0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:23.351002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a542f2-6b8f-46b6-acfc-894e62b179f0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:23.352596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 554/1107] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:23.359361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:23.359828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:23.359963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:23.389115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:23.391131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:23.480159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 554/1108] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:23.752427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a542f2-6b8f-46b6-acfc-894e62b179f0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:23.872337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a542f2-6b8f-46b6-acfc-894e62b179f0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:23.908223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cda6d4-0ca5-4e56-bee5-e83debe5aafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:23.908848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda6d4-0ca5-4e56-bee5-e83debe5aafe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:23.909226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda6d4-0ca5-4e56-bee5-e83debe5aafe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:23.910032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cda6d4-0ca5-4e56-bee5-e83debe5aafe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:23.911190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cda6d4-0ca5-4e56-bee5-e83debe5aafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:23.912626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cda6d4-0ca5-4e56-bee5-e83debe5aafe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:23.914228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 555/1109] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:23.920362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:24.004313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:24.026520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd1d18d-b31e-4e96-9028-25d7daa635ab None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:24.062887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371d2184-ad74-442d-b2cc-46dc64d015ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:24.063501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2184-ad74-442d-b2cc-46dc64d015ef None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:24.063778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2184-ad74-442d-b2cc-46dc64d015ef None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:24.064048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d2184-ad74-442d-b2cc-46dc64d015ef None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:24.064697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d2184-ad74-442d-b2cc-46dc64d015ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:24.066142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371d2184-ad74-442d-b2cc-46dc64d015ef None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:24.089240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 555/1110] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:24:23 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:26.084256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cca1c0-ce97-44e9-a026-56585a6738f5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:26.084970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cca1c0-ce97-44e9-a026-56585a6738f5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:26.084970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cca1c0-ce97-44e9-a026-56585a6738f5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:26.086120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 556/1111] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:26.092678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:26.093137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:26.093478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:26.115619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:26.116808 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:26.186519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 556/1112] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:26.454630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cca1c0-ce97-44e9-a026-56585a6738f5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:26.482775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:26.555894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cca1c0-ce97-44e9-a026-56585a6738f5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:26.589722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:26.590377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:26.590656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:26.590926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] PATH_INFO: `/v3/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:26.592320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] RBAC: Authorizing `identity:get_role(role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:26.593694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:26.595056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 557/1113] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:26 2022] GET /identity/v3/roles/712bb84fd46d48f28cbe60cbb255213f => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:26.601473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:26.670403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:26.707514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683c87be-a780-4fda-9e92-465fa734c91c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:26.741751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:26.742440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:26.742720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:26.742990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] PATH_INFO: `/v3/users/9c0655b2df29422b84377ad106a8d917` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:26.756010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] RBAC: Authorizing `identity:get_user(user_id=9c0655b2df29422b84377ad106a8d917)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:26.757448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:26.759092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 557/1114] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:26 2022] GET /identity/v3/users/9c0655b2df29422b84377ad106a8d917 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:26.764946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:26.857582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:26.864045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68711e9c-bc16-4824-b758-d549dc0364d1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:26.896775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:26.897368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:26.897640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:26.897906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:26.967550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:26.968906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:27.058415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:27.064931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 558/1115] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:24:27.071569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:27.175642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eef512-58e1-4eab-a725-d2a5c0f74e13 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:27.198439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:27.209306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d909993-df0d-4348-a9fa-138f0344df87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:27.209939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d909993-df0d-4348-a9fa-138f0344df87 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:27.210249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d909993-df0d-4348-a9fa-138f0344df87 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:27.210514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d909993-df0d-4348-a9fa-138f0344df87 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:27.211606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d909993-df0d-4348-a9fa-138f0344df87 None admin] RBAC: Authorizing `identity:get_project(project_id=373d1e6c3f8041e6be98b77d917f3161)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:27.213023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d909993-df0d-4348-a9fa-138f0344df87 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:27.214912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 558/1116] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:27 2022] GET /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:27.221190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:27.317960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a950b-867e-4a60-a923-3b1dc06d3849 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:27.354029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:27.354659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:27.354932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:27.355199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:27.355911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] RBAC: Adding query filter params (role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=373d1e6c3f8041e6be98b77d917f3161, user.id=9c0655b2df29422b84377ad106a8d917) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:27.356213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:27.357645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc6cfb-7095-4e51-8ce3-1d7d17f488e8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:27.363051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 559/1117] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:27 2022] GET /identity/v3/role_assignments?user.id=9c0655b2df29422b84377ad106a8d917&scope.project.id=373d1e6c3f8041e6be98b77d917f3161&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:29.226899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131f204-b610-4e96-9e29-1948cc889dfb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:29.227436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131f204-b610-4e96-9e29-1948cc889dfb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:29.227528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6131f204-b610-4e96-9e29-1948cc889dfb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:29.228925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 559/1118] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:29.234413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:29.234889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:29.234994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:29.253668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:29.255019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:29.312028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:29.324391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 560/1119] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:29.597915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131f204-b610-4e96-9e29-1948cc889dfb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:29.704470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6131f204-b610-4e96-9e29-1948cc889dfb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:29.738689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:29.739353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:29.739594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:29.739866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] PATH_INFO: `/v3/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:29.741307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] RBAC: Authorizing `identity:get_role(role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:29.742910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:29.744251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 560/1120] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:29 2022] GET /identity/v3/roles/712bb84fd46d48f28cbe60cbb255213f => generated 223 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:29.752452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:29.866016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a18b3b-5641-4398-a3ee-2d89d7d6e717 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:29.904230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:29.904865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:29.905147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:29.905418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] PATH_INFO: `/v3/users/9c0655b2df29422b84377ad106a8d917` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:29.906592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] RBAC: Authorizing `identity:get_user(user_id=9c0655b2df29422b84377ad106a8d917)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:29.908191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:29.909901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 561/1121] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:29 2022] GET /identity/v3/users/9c0655b2df29422b84377ad106a8d917 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:29.916417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:29.980896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:30.023353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566d668d-e86e-4f1c-a13f-9ff62556aca5 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:30.062079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3857aae2-f3fc-4cfa-b9f3-4d004d9fd1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:30.062719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857aae2-f3fc-4cfa-b9f3-4d004d9fd1f0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:30.063133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857aae2-f3fc-4cfa-b9f3-4d004d9fd1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:30.063493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3857aae2-f3fc-4cfa-b9f3-4d004d9fd1f0 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:30.064949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857aae2-f3fc-4cfa-b9f3-4d004d9fd1f0 None admin] RBAC: Authorizing `identity:get_project(project_id=373d1e6c3f8041e6be98b77d917f3161)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:30.066583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3857aae2-f3fc-4cfa-b9f3-4d004d9fd1f0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:30.068924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 561/1122] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:29 2022] GET /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:30.076068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:30.142903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:30.187509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fae775-a788-4751-990b-30df9ba22cbd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:30.223868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8dfdc-ed4a-450e-a82d-3be1dd964495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:30.224524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8dfdc-ed4a-450e-a82d-3be1dd964495 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:30.224799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8dfdc-ed4a-450e-a82d-3be1dd964495 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:30.225068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8dfdc-ed4a-450e-a82d-3be1dd964495 None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161/users/9c0655b2df29422b84377ad106a8d917/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:30.227225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8dfdc-ed4a-450e-a82d-3be1dd964495 None admin] RBAC: Authorizing `identity:create_grant(project_id=373d1e6c3f8041e6be98b77d917f3161, user_id=9c0655b2df29422b84377ad106a8d917, role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:30.228662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8dfdc-ed4a-450e-a82d-3be1dd964495 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:30.244013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 562/1123] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:24:30 2022] PUT /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161/users/9c0655b2df29422b84377ad106a8d917/roles/712bb84fd46d48f28cbe60cbb255213f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:24:32.170726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4074836-0b61-4dc1-be82-76cbbe85657d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:32.171242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4074836-0b61-4dc1-be82-76cbbe85657d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:32.171242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4074836-0b61-4dc1-be82-76cbbe85657d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:32.172693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 562/1124] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:32.178625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:32.179065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:32.179168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:32.197388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:32.198627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:32.266442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 563/1125] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:32.532357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4074836-0b61-4dc1-be82-76cbbe85657d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:32.631887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4074836-0b61-4dc1-be82-76cbbe85657d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:32.674063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:32.681942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:32.682597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:32.682874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:32.683143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] PATH_INFO: `/v3/roles/712bb84fd46d48f28cbe60cbb255213f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:32.684513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] RBAC: Authorizing `identity:get_role(role_id=712bb84fd46d48f28cbe60cbb255213f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:32.685870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:32.687228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 563/1126] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:32 2022] GET /identity/v3/roles/712bb84fd46d48f28cbe60cbb255213f => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:32.693130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:32.742785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:32.804615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e4b7c8-8433-4ad7-b852-d1ad5d6d098a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:32.838591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:32.839220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:32.839496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:32.839767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] PATH_INFO: `/v3/users/9c0655b2df29422b84377ad106a8d917` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:32.840887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] RBAC: Authorizing `identity:get_user(user_id=9c0655b2df29422b84377ad106a8d917)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:32.842388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:32.843942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 564/1127] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:32 2022] GET /identity/v3/users/9c0655b2df29422b84377ad106a8d917 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:32.850372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:32.954099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab819660-c238-4288-835e-c51754ebb722 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:32.989557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:32.990156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:32.990464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:32.990731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:33.052289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:33.059339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:33.060729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:33.159995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 564/1128] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:32 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:24:33.166333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:33.263494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0036f6e9-4ba0-4c07-ae7d-7442669e0078 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:33.280406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:33.301206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cf8846-20b1-4e3c-9bd7-a61d8353331f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:33.301828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf8846-20b1-4e3c-9bd7-a61d8353331f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:33.302102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf8846-20b1-4e3c-9bd7-a61d8353331f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:33.302395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf8846-20b1-4e3c-9bd7-a61d8353331f None admin] PATH_INFO: `/v3/projects/373d1e6c3f8041e6be98b77d917f3161` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:33.303738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf8846-20b1-4e3c-9bd7-a61d8353331f None admin] RBAC: Authorizing `identity:get_project(project_id=373d1e6c3f8041e6be98b77d917f3161)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:33.305898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cf8846-20b1-4e3c-9bd7-a61d8353331f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:33.308326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 565/1129] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:33 2022] GET /identity/v3/projects/373d1e6c3f8041e6be98b77d917f3161 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:33.314752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:33.409391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:33.415716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4f6332-0a7f-4aed-b6f3-beac4bb13976 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:33.450011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:33.450634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:33.450914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:33.451181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:33.451896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] RBAC: Adding query filter params (role.id=712bb84fd46d48f28cbe60cbb255213f, scope.project.id=373d1e6c3f8041e6be98b77d917f3161, user.id=9c0655b2df29422b84377ad106a8d917) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:33.452201 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:33.453621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b36773-dd57-4001-afb5-1d268ccd9eff None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:33.459283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 565/1130] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:33 2022] GET /identity/v3/role_assignments?user.id=9c0655b2df29422b84377ad106a8d917&scope.project.id=373d1e6c3f8041e6be98b77d917f3161&role.id=712bb84fd46d48f28cbe60cbb255213f => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:35.387451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338bf1-0f9a-476e-9495-0874312b2819 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:35.387958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338bf1-0f9a-476e-9495-0874312b2819 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:35.387958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a338bf1-0f9a-476e-9495-0874312b2819 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:35.389221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 566/1131] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:35.395014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:35.395450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:35.395563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:35.413938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:35.415181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:35.481929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 566/1132] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:24:35.750345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a338bf1-0f9a-476e-9495-0874312b2819 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:35.852883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a338bf1-0f9a-476e-9495-0874312b2819 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:35.885912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197cfbc7-85c1-481c-ac1e-78263c6874a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:35.886560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cfbc7-85c1-481c-ac1e-78263c6874a5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:35.886840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cfbc7-85c1-481c-ac1e-78263c6874a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:35.887109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197cfbc7-85c1-481c-ac1e-78263c6874a5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:35.888224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cfbc7-85c1-481c-ac1e-78263c6874a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:35.889662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197cfbc7-85c1-481c-ac1e-78263c6874a5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:35.891267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 567/1133] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:35.897657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:35.987600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:35.997681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7aad31-c0b9-454f-be00-a11c23dbd957 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:36.032597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f01f2df-5618-45ef-902e-0eef97dbf81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:36.033121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01f2df-5618-45ef-902e-0eef97dbf81d None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:36.033478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01f2df-5618-45ef-902e-0eef97dbf81d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:36.033667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f01f2df-5618-45ef-902e-0eef97dbf81d None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:36.034346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01f2df-5618-45ef-902e-0eef97dbf81d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:36.036404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f01f2df-5618-45ef-902e-0eef97dbf81d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:36.063200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 567/1134] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:24:35 2022] POST /identity/v3/projects => generated 311 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:37.977416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2a6ad-ca99-4725-8792-61576872d35d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:37.977981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2a6ad-ca99-4725-8792-61576872d35d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:37.977981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba2a6ad-ca99-4725-8792-61576872d35d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:37.979319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 568/1135] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:24:37.984954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:37.985379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:37.985496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:38.004400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:38.005598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:38.082794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 568/1136] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:38.348354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2a6ad-ca99-4725-8792-61576872d35d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:38.403358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:38.459194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba2a6ad-ca99-4725-8792-61576872d35d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:38.494075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ad415-1215-4b36-956b-c737af0323df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:38.494724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad415-1215-4b36-956b-c737af0323df None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:38.495003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad415-1215-4b36-956b-c737af0323df None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:38.495272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ad415-1215-4b36-956b-c737af0323df None admin] PATH_INFO: `/v3/domains/default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:38.496385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ad415-1215-4b36-956b-c737af0323df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:38.497819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0ad415-1215-4b36-956b-c737af0323df None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:38.499431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 569/1137] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:24:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:38.505364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:38.575337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:38.606003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bcc91f-6ba6-4179-a6a8-34c12c44fc7c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:38.639939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca877e3-a68e-4fe8-a0a8-a2ce4f7e5b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:38.640638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca877e3-a68e-4fe8-a0a8-a2ce4f7e5b23 None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:38.640904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca877e3-a68e-4fe8-a0a8-a2ce4f7e5b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:38.641175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca877e3-a68e-4fe8-a0a8-a2ce4f7e5b23 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:38.641822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca877e3-a68e-4fe8-a0a8-a2ce4f7e5b23 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:38.643271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca877e3-a68e-4fe8-a0a8-a2ce4f7e5b23 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:38.665547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 569/1138] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:24:38 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:24:40.609912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b521e-a918-4442-a57c-1afde843752c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:40.610506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b521e-a918-4442-a57c-1afde843752c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:40.610506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b521e-a918-4442-a57c-1afde843752c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:40.611793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 570/1139] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:40.617618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:40.618090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:40.618209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:40.636434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:40.637622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:40.701623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 570/1140] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:40.972968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b521e-a918-4442-a57c-1afde843752c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:41.071931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b521e-a918-4442-a57c-1afde843752c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:41.091545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:41.107731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:41.108160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:41.108491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:41.108763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:41.119409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:41.124045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:41.125261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:41.126249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 571/1141] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:24:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:41.131333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:41.188960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:41.230544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96a826-8b29-45b3-a7f2-01f5c97f2192 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:41.263832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:41.264431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:41.264710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:41.264982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:41.265649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:41.265950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:41.267347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:41.275843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 571/1142] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:24:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:41.282012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:41.387692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71072a53-a56f-4d64-99c0-618833ea764e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:41.424074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:41.424868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:41.424995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:41.425316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] PATH_INFO: `/v3/users/a4f71cb0e946465fa1e18eee115f27dd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:41.439256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f71cb0e946465fa1e18eee115f27dd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:41.440710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:41.442966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 572/1143] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:41 2022] GET /identity/v3/users/a4f71cb0e946465fa1e18eee115f27dd => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:41.448957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:41.549126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da5239-fa7d-48fa-8d02-c67e3c4cbbfb None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:41.568480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:41.584097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:41.584710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:41.584993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:41.585266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:41.653494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:41.655072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:41.751117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 572/1144] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:41 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:24:41.757729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:41.797120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:41.856946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eb44dc-c9c2-409c-98d7-c6d3dcde88a4 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:41.890821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c76a47-73c2-49c2-802b-5eecac7e72e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:41.891446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c76a47-73c2-49c2-802b-5eecac7e72e8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:41.891724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c76a47-73c2-49c2-802b-5eecac7e72e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:41.891996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c76a47-73c2-49c2-802b-5eecac7e72e8 None admin] PATH_INFO: `/v3/projects/71022699c86544ec9d837ddea967cff3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:41.893112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c76a47-73c2-49c2-802b-5eecac7e72e8 None admin] RBAC: Authorizing `identity:get_project(project_id=71022699c86544ec9d837ddea967cff3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:41.894607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c76a47-73c2-49c2-802b-5eecac7e72e8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:41.896234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 573/1145] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:41 2022] GET /identity/v3/projects/71022699c86544ec9d837ddea967cff3 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:41.902570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:41.923161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:42.000493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ff48c-7c73-4bb8-8be9-76fb8d70fe35 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:42.033143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:42.033736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:42.034008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:42.034299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:42.035551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=71022699c86544ec9d837ddea967cff3, user.id=a4f71cb0e946465fa1e18eee115f27dd) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:42.035849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:42.037248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2a3c1f-18f4-4288-8ae5-754ebd4817a9 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:42.042653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 573/1146] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:41 2022] GET /identity/v3/role_assignments?user.id=a4f71cb0e946465fa1e18eee115f27dd&scope.project.id=71022699c86544ec9d837ddea967cff3&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:43.908955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081221f-d1a0-47cd-8ab4-45e3f44d18b6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:43.909493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081221f-d1a0-47cd-8ab4-45e3f44d18b6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:43.909493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1081221f-d1a0-47cd-8ab4-45e3f44d18b6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:43.911457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 574/1147] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:43.917773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:43.918217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:43.918327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:43.935737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:43.936908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:44.001408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 574/1148] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:44.270892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081221f-d1a0-47cd-8ab4-45e3f44d18b6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:44.370634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1081221f-d1a0-47cd-8ab4-45e3f44d18b6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:44.377765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:44.404013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:44.404630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:44.404910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:44.405179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:44.415627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:44.420539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:44.421718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:44.422862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 575/1149] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:24:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:44.428220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:44.443335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:44.523644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c374cf2-d947-4e06-beaa-c699e7b7440b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:44.557039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:44.557641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:44.557918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:44.558221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:44.558878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:44.559179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:44.560543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:44.568391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 575/1150] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:24:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:44.576376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:44.679448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d921a82-9ab6-46c9-b506-1c964e192150 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:44.712522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:44.713142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:44.713423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:44.713700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] PATH_INFO: `/v3/users/a4f71cb0e946465fa1e18eee115f27dd` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:44.714844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f71cb0e946465fa1e18eee115f27dd)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:44.716277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:44.717859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 576/1151] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:44 2022] GET /identity/v3/users/a4f71cb0e946465fa1e18eee115f27dd => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:24:44.724255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:44.801471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:44.823458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c19215-e582-4705-9d84-0594ddb65443 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:44.856313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f64241-e06a-4290-91f1-c527a426f396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:44.856878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f64241-e06a-4290-91f1-c527a426f396 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:44.857158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f64241-e06a-4290-91f1-c527a426f396 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:44.857426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f64241-e06a-4290-91f1-c527a426f396 None admin] PATH_INFO: `/v3/projects/71022699c86544ec9d837ddea967cff3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:44.858547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f64241-e06a-4290-91f1-c527a426f396 None admin] RBAC: Authorizing `identity:get_project(project_id=71022699c86544ec9d837ddea967cff3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:44.859965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f64241-e06a-4290-91f1-c527a426f396 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:44.861579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 576/1152] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:44 2022] GET /identity/v3/projects/71022699c86544ec9d837ddea967cff3 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:44.867619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:44.900967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:44.966900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e382ca11-55b1-45cb-8596-137bf68be598 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:45.001081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c5aa57-c9fe-41e9-8e75-fa4871e063a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:45.001738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5aa57-c9fe-41e9-8e75-fa4871e063a0 None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:45.002017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5aa57-c9fe-41e9-8e75-fa4871e063a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:45.002319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c5aa57-c9fe-41e9-8e75-fa4871e063a0 None admin] PATH_INFO: `/v3/projects/71022699c86544ec9d837ddea967cff3/users/a4f71cb0e946465fa1e18eee115f27dd/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:45.004746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5aa57-c9fe-41e9-8e75-fa4871e063a0 None admin] RBAC: Authorizing `identity:create_grant(project_id=71022699c86544ec9d837ddea967cff3, user_id=a4f71cb0e946465fa1e18eee115f27dd, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:45.006198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c5aa57-c9fe-41e9-8e75-fa4871e063a0 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:45.021457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 577/1153] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:24:44 2022] PUT /identity/v3/projects/71022699c86544ec9d837ddea967cff3/users/a4f71cb0e946465fa1e18eee115f27dd/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:24:46.880515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d541b4-82db-4da1-abb8-442dcac270cb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:46.881049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d541b4-82db-4da1-abb8-442dcac270cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:46.881049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d541b4-82db-4da1-abb8-442dcac270cb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:46.882439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 577/1154] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:46.887733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:46.888185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:46.888323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:46.906322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:46.907507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:46.974383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 578/1155] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:47.245581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d541b4-82db-4da1-abb8-442dcac270cb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:47.345560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d541b4-82db-4da1-abb8-442dcac270cb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:47.378580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:47.379218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:47.379492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:47.379759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:47.390405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:24:47.395017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:47.396110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:24:47.399315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 578/1156] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:24:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:47.402151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:47.427281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:47.500423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382cd2d-9203-4180-bd36-9d38f36da897 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:47.533041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:47.533640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:47.533918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:47.534213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:47.534875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:47.535686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:47.537056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:47.545232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 579/1157] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:24:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:47.551673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:47.555434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:47.649403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bec9d5c-4d5a-4193-8429-12023450cccd None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:47.683048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:47.683675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:47.683950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:47.684224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] PATH_INFO: `/v3/users/a4f71cb0e946465fa1e18eee115f27dd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:47.685329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] RBAC: Authorizing `identity:get_user(user_id=a4f71cb0e946465fa1e18eee115f27dd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:47.686784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:47.688361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 579/1158] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:24:47 2022] GET /identity/v3/users/a4f71cb0e946465fa1e18eee115f27dd => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:47.694332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:47.777888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:47.793664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2875b22b-c862-4361-b75f-eaa9016eaaa1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:47.826456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:47.827056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:47.827337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:47.827610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:47.893462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:47.894932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:47.976294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:47.991731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 580/1159] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:47 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:24:47.997602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:48.067609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:48.098519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846295d5-b33e-4b5c-9c04-e3d73f90c371 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:48.132986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2829e5e0-2876-468d-af67-f947fa3e0cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:48.133614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e5e0-2876-468d-af67-f947fa3e0cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:48.133893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e5e0-2876-468d-af67-f947fa3e0cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:48.134166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2829e5e0-2876-468d-af67-f947fa3e0cd8 None admin] PATH_INFO: `/v3/projects/71022699c86544ec9d837ddea967cff3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:48.135300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e5e0-2876-468d-af67-f947fa3e0cd8 None admin] RBAC: Authorizing `identity:get_project(project_id=71022699c86544ec9d837ddea967cff3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:48.136722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2829e5e0-2876-468d-af67-f947fa3e0cd8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:48.138375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 580/1160] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:24:47 2022] GET /identity/v3/projects/71022699c86544ec9d837ddea967cff3 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:48.145163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:48.243885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d68945-fed2-4f1d-b202-c204bbd3ef9d None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:48.278544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:48.279145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:48.279418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:48.279687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:48.280402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=71022699c86544ec9d837ddea967cff3, user.id=a4f71cb0e946465fa1e18eee115f27dd) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:48.280705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:48.282129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851d5aad-04ca-4e21-a30e-85e40f72e380 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:48.288007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 581/1161] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:24:48 2022] GET /identity/v3/role_assignments?user.id=a4f71cb0e946465fa1e18eee115f27dd&scope.project.id=71022699c86544ec9d837ddea967cff3&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:50.201995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986e42-6a72-49f3-9116-a2938fbb22da None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:50.202572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986e42-6a72-49f3-9116-a2938fbb22da None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:50.202572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17986e42-6a72-49f3-9116-a2938fbb22da None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:50.203874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 581/1162] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:50.208994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:50.209415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:50.209509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:50.227457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:50.228631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:50.273138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:50.294874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 582/1163] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:50.560447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17986e42-6a72-49f3-9116-a2938fbb22da None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:50.660415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17986e42-6a72-49f3-9116-a2938fbb22da None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:50.693191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:50.693786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:50.694066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:50.694374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:50.704394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:50.760051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:50.761497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:50.857588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 582/1164] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:24:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:24:50.863609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:50.961397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c9a6a2-6f9c-4b24-9469-dcff7c9dc882 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:50.995670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:50.996293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:50.996571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:50.996839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:51.003343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:51.004777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-0ac24b8e-20e7-451c-8a43-72e64bb91464 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:24:51.015778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:24:51.016997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:24:51.018071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 583/1165] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:24:50 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:24:51.023072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:51.065021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:51.121526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014ec56-fddb-4926-9bbc-313454e695f3 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:51.153897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:51.154519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:51.154799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:51.155070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:51.155733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:24:51.156031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:51.157455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb67791-5051-4a63-842f-60bc9ebfc1af None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:51.165005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 583/1166] 10.222.0.91 () {56 vars in 1115 bytes} [Wed Aug 24 03:24:51 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:53.029982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f03bdc-746c-44cd-8d68-de2d99a22c90 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:53.030565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f03bdc-746c-44cd-8d68-de2d99a22c90 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:53.030565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f03bdc-746c-44cd-8d68-de2d99a22c90 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:53.031878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 584/1167] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:53.037157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266c342-de50-406f-a457-3ef9caa3eceb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:53.037610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266c342-de50-406f-a457-3ef9caa3eceb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:53.037703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8266c342-de50-406f-a457-3ef9caa3eceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:53.056131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8266c342-de50-406f-a457-3ef9caa3eceb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:53.057318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8266c342-de50-406f-a457-3ef9caa3eceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:53.122644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 584/1168] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:53.391608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f03bdc-746c-44cd-8d68-de2d99a22c90 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:53.433827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:53.491026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f03bdc-746c-44cd-8d68-de2d99a22c90 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:53.524574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cefa31f-9765-4ad0-bb27-0d998c9efea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:53.525173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefa31f-9765-4ad0-bb27-0d998c9efea3 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:53.525453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefa31f-9765-4ad0-bb27-0d998c9efea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:53.525721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cefa31f-9765-4ad0-bb27-0d998c9efea3 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:53.526415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefa31f-9765-4ad0-bb27-0d998c9efea3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:53.527787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cefa31f-9765-4ad0-bb27-0d998c9efea3 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:53.544790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 585/1169] 10.222.0.91 () {60 vars in 1138 bytes} [Wed Aug 24 03:24:53 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:55.452275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4a9d6-bd9e-4717-8850-77523d3d9695 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:55.452805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4a9d6-bd9e-4717-8850-77523d3d9695 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:55.452805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf4a9d6-bd9e-4717-8850-77523d3d9695 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:55.454168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 585/1170] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:55.459455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:55.459898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:55.459996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:55.478065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:55.479278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:55.543663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 586/1171] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:55.811970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4a9d6-bd9e-4717-8850-77523d3d9695 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:55.846017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:55.910568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf4a9d6-bd9e-4717-8850-77523d3d9695 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:55.943945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37203af-c22f-46a0-b868-df60261f27fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:55.944562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37203af-c22f-46a0-b868-df60261f27fb None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:55.944834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37203af-c22f-46a0-b868-df60261f27fb None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:55.945100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37203af-c22f-46a0-b868-df60261f27fb None admin] PATH_INFO: `/v3/domains/0e1cac9578a143c19375fee82ed47cef` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:55.955132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37203af-c22f-46a0-b868-df60261f27fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=0e1cac9578a143c19375fee82ed47cef)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:55.956547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37203af-c22f-46a0-b868-df60261f27fb None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:55.958661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 586/1172] 10.222.0.91 () {56 vars in 1183 bytes} [Wed Aug 24 03:24:55 2022] GET /identity/v3/domains/0e1cac9578a143c19375fee82ed47cef => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:55.964499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:55.993942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:56.064178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d65107-749f-47de-9a77-e1cf1117a648 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:56.097059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35942056-4715-4995-bafd-d75ad0739e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:56.097663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35942056-4715-4995-bafd-d75ad0739e4b None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:56.097941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35942056-4715-4995-bafd-d75ad0739e4b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:56.098236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35942056-4715-4995-bafd-d75ad0739e4b None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:56.099494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35942056-4715-4995-bafd-d75ad0739e4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:56.100936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35942056-4715-4995-bafd-d75ad0739e4b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:56.127416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 587/1173] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:24:55 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:24:58.062412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72603a-5d14-4d57-9c51-59df434cd9c6 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:58.062999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72603a-5d14-4d57-9c51-59df434cd9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:58.062999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da72603a-5d14-4d57-9c51-59df434cd9c6 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:58.064295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 587/1174] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:24:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:24:58.069651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:58.070074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:58.070211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:58.088391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:24:58.089579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:24:58.158321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 588/1175] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:24:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:24:58.427687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72603a-5d14-4d57-9c51-59df434cd9c6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:58.527877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:58.536172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da72603a-5d14-4d57-9c51-59df434cd9c6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:58.570961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e285aed5-50bd-434a-ae08-fbcc45c5d66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:58.571595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285aed5-50bd-434a-ae08-fbcc45c5d66c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:58.571872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285aed5-50bd-434a-ae08-fbcc45c5d66c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:58.572170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e285aed5-50bd-434a-ae08-fbcc45c5d66c None admin] PATH_INFO: `/v3/domains/0e1cac9578a143c19375fee82ed47cef` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:58.573828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285aed5-50bd-434a-ae08-fbcc45c5d66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=0e1cac9578a143c19375fee82ed47cef)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:58.575371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e285aed5-50bd-434a-ae08-fbcc45c5d66c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:58.577031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 588/1176] 10.222.0.91 () {56 vars in 1183 bytes} [Wed Aug 24 03:24:58 2022] GET /identity/v3/domains/0e1cac9578a143c19375fee82ed47cef => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:24:58.584740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:24:58.599822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:24:58.695290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf16575-e4d9-4b1c-80b3-c797c2a165cb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:24:58.731637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a12ca-e15f-4107-9dad-380ab76e11a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:24:58.732246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a12ca-e15f-4107-9dad-380ab76e11a5 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:24:58.732524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a12ca-e15f-4107-9dad-380ab76e11a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:24:58.732793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a12ca-e15f-4107-9dad-380ab76e11a5 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:24:58.733436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a12ca-e15f-4107-9dad-380ab76e11a5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:24:58.734914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a12ca-e15f-4107-9dad-380ab76e11a5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:24:58.756905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 589/1177] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:24:58 2022] POST /identity/v3/users => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:25:00.737711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc97b1-a891-4760-b301-06015895c8bd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:00.738307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc97b1-a891-4760-b301-06015895c8bd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:00.738307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc97b1-a891-4760-b301-06015895c8bd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:00.739663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 589/1178] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:00.745058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:00.745471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:00.745600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:00.763730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:00.764933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:00.826745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:00.833558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 590/1179] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:25:01.102130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc97b1-a891-4760-b301-06015895c8bd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:01.206120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc97b1-a891-4760-b301-06015895c8bd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:01.240257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:01.240893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:01.241167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:01.241435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:01.252703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:25:01.258086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:25:01.259537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:25:01.260717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 590/1180] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:25:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:25:01.265447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:01.367924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd463b02-ae84-4870-bc20-7ac7ebd0fa87 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:01.401196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:01.401803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:01.402084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:01.402381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:01.403052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:25:01.403353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:01.404729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:01.413145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 591/1181] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:25:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:01.420234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:01.445786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:01.519910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249f59a-805d-4be1-8872-f794c919c414 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:01.553175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:01.553799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:01.554080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:01.554381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] PATH_INFO: `/v3/users/1899613c5bc74ff29464e5ffeca5c044` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:01.567074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] RBAC: Authorizing `identity:get_user(user_id=1899613c5bc74ff29464e5ffeca5c044)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:01.568502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:01.570088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 591/1182] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:25:01 2022] GET /identity/v3/users/1899613c5bc74ff29464e5ffeca5c044 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:01.576079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:01.637148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:01.678390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20015cb9-e24a-4c52-a6ba-6c18b5f9efdc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:01.710320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:01.710921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:01.711201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:01.711472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:01.776905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:01.778386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:01.833601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:01.874036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 592/1183] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:25:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:25:01.880034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:01.958401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:01.977611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab9e47-ec09-45f7-b4a3-e3c9c51e5c23 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:02.009761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5357dd-f038-4712-a030-1422afbda3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:02.010400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5357dd-f038-4712-a030-1422afbda3b1 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:02.010688 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5357dd-f038-4712-a030-1422afbda3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:02.010959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5357dd-f038-4712-a030-1422afbda3b1 None admin] PATH_INFO: `/v3/projects/1a8443c02fa74f019ac29c457342fd4b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:02.012063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5357dd-f038-4712-a030-1422afbda3b1 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8443c02fa74f019ac29c457342fd4b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:02.013478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5357dd-f038-4712-a030-1422afbda3b1 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:02.015153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 592/1184] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:25:01 2022] GET /identity/v3/projects/1a8443c02fa74f019ac29c457342fd4b => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:25:02.021694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:02.119033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed6817a-5b06-4457-8b47-3b6ba1dc909b None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:02.153001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:02.153597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:02.153877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:02.154150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:02.154903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=1a8443c02fa74f019ac29c457342fd4b, user.id=1899613c5bc74ff29464e5ffeca5c044) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:25:02.155210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:02.156648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a6b2d6-a501-4706-99ad-3fb85bfff066 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:02.162481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 593/1185] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:25:02 2022] GET /identity/v3/role_assignments?user.id=1899613c5bc74ff29464e5ffeca5c044&scope.project.id=1a8443c02fa74f019ac29c457342fd4b&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:04.027336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1b36a-d5bc-49e8-b5ba-3c253abf637a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:04.027899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1b36a-d5bc-49e8-b5ba-3c253abf637a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:04.027899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1b36a-d5bc-49e8-b5ba-3c253abf637a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:04.029206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 593/1186] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:25:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:04.034403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:04.034821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:04.034968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:04.053550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:04.054762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:04.072154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:04.122139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 594/1187] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:25:04.404166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1b36a-d5bc-49e8-b5ba-3c253abf637a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:04.515462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1b36a-d5bc-49e8-b5ba-3c253abf637a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:04.553216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:04.553857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:04.554133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:04.554502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:04.564832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:25:04.569593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:25:04.570813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:25:04.571927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 594/1188] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:25:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:25:04.577943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:04.678622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46514d-0bfb-4476-bcb2-9a9e8d1bee5e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:04.715445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:04.716048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:04.716328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:04.716596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:04.717270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:25:04.717574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:04.718983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:04.727286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 595/1189] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:25:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:04.733211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:04.744734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:04.835181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07223151-33ff-4feb-86f0-49cacfd16f0f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:04.870440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:04.871065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:04.871338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:04.871601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] PATH_INFO: `/v3/users/1899613c5bc74ff29464e5ffeca5c044` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:04.872717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] RBAC: Authorizing `identity:get_user(user_id=1899613c5bc74ff29464e5ffeca5c044)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:04.874132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:04.875753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 595/1190] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:25:04 2022] GET /identity/v3/users/1899613c5bc74ff29464e5ffeca5c044 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:04.882084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:04.901882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:04.988756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1faab6-0ee1-407d-95e2-e0384ca478b6 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:05.021790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af691b-f226-4bea-84cd-49dcb38e0cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:05.022438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af691b-f226-4bea-84cd-49dcb38e0cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:05.022722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af691b-f226-4bea-84cd-49dcb38e0cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:05.022994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af691b-f226-4bea-84cd-49dcb38e0cc2 None admin] PATH_INFO: `/v3/projects/1a8443c02fa74f019ac29c457342fd4b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:05.024103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af691b-f226-4bea-84cd-49dcb38e0cc2 None admin] RBAC: Authorizing `identity:get_project(project_id=1a8443c02fa74f019ac29c457342fd4b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:05.025538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af691b-f226-4bea-84cd-49dcb38e0cc2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:05.027194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 596/1191] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:25:04 2022] GET /identity/v3/projects/1a8443c02fa74f019ac29c457342fd4b => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:05.033305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:05.112114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:05.135713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe534c5-8eba-482e-a5cb-4992d6fd82b4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:05.170465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ced9eab-468a-4d54-a6ed-d5c256d58f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:05.171118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9eab-468a-4d54-a6ed-d5c256d58f09 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:05.171396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9eab-468a-4d54-a6ed-d5c256d58f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:05.171668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ced9eab-468a-4d54-a6ed-d5c256d58f09 None admin] PATH_INFO: `/v3/projects/1a8443c02fa74f019ac29c457342fd4b/users/1899613c5bc74ff29464e5ffeca5c044/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:05.173543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced9eab-468a-4d54-a6ed-d5c256d58f09 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a8443c02fa74f019ac29c457342fd4b, user_id=1899613c5bc74ff29464e5ffeca5c044, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:05.174992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ced9eab-468a-4d54-a6ed-d5c256d58f09 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:05.187826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 596/1192] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:25:05 2022] PUT /identity/v3/projects/1a8443c02fa74f019ac29c457342fd4b/users/1899613c5bc74ff29464e5ffeca5c044/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:25:07.097323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c74ddf-eaa4-45a0-9871-c5f7e1b5434e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:07.097866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c74ddf-eaa4-45a0-9871-c5f7e1b5434e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:07.097972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c74ddf-eaa4-45a0-9871-c5f7e1b5434e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:07.099462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 597/1193] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:25:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:25:07.104759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:07.105195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:07.105474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:07.124078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:25:07.125283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:25:07.196054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 597/1194] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:25:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:25:07.459709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c74ddf-eaa4-45a0-9871-c5f7e1b5434e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:07.544413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:07.560728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c74ddf-eaa4-45a0-9871-c5f7e1b5434e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:07.595368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:07.595996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:07.596271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:07.596537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:07.606859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:25:07.611485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:25:07.612596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:25:07.613610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 598/1195] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:25:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:25:07.618926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:07.682737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:07.718028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a997ef5-843c-47fd-8115-fe903a4813a0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:07.750414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:07.751016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:07.751295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:07.751565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:07.752222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:25:07.752522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:07.754465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:07.762316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 598/1196] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:25:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:07.768181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:07.868977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17884a27-8243-4556-b427-440c5f3d531a None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:07.903031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:07.903643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:07.903917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:07.904185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] PATH_INFO: `/v3/users/1899613c5bc74ff29464e5ffeca5c044` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:07.905288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] RBAC: Authorizing `identity:get_user(user_id=1899613c5bc74ff29464e5ffeca5c044)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:07.906747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:07.908312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 599/1197] 10.222.0.91 () {56 vars in 1177 bytes} [Wed Aug 24 03:25:07 2022] GET /identity/v3/users/1899613c5bc74ff29464e5ffeca5c044 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:07.914336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:08.013766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b85b2b6-7a12-4384-8f97-2cabbdeaa40d None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:08.041341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:08.049405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:08.050550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:08.050831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:08.051097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:08.117798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:08.119275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:08.219404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 599/1198] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:25:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:25:08.226002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:08.241677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:08.328191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafeebc5-d51d-4743-b04a-c20d88309b12 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:08.369583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b2630-be3b-4377-8316-61a56e033b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:08.370218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b2630-be3b-4377-8316-61a56e033b7c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:08.370492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b2630-be3b-4377-8316-61a56e033b7c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:08.370757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b2630-be3b-4377-8316-61a56e033b7c None admin] PATH_INFO: `/v3/projects/1a8443c02fa74f019ac29c457342fd4b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:08.371861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b2630-be3b-4377-8316-61a56e033b7c None admin] RBAC: Authorizing `identity:get_project(project_id=1a8443c02fa74f019ac29c457342fd4b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:08.373298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b2630-be3b-4377-8316-61a56e033b7c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:08.374948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 600/1199] 10.222.0.91 () {56 vars in 1186 bytes} [Wed Aug 24 03:25:08 2022] GET /identity/v3/projects/1a8443c02fa74f019ac29c457342fd4b => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:08.381333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:25:08.408485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:08.486274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f68395-c3cb-491b-82f4-0d991037e7a7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:25:08.523229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:25:08.523839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:25:08.524121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:25:08.524393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:25:08.525122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=1a8443c02fa74f019ac29c457342fd4b, user.id=1899613c5bc74ff29464e5ffeca5c044) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:25:08.525429 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:25:08.526897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e8fd6f-1eec-4cdc-8f8a-e177d69f0631 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:25:08.532832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 600/1200] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:25:08 2022] GET /identity/v3/role_assignments?user.id=1899613c5bc74ff29464e5ffeca5c044&scope.project.id=1a8443c02fa74f019ac29c457342fd4b&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:25:18.365353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=60 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:25:18.530164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=54 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:06.455442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c6a6f-4e19-41eb-ae78-ab1a35ab3bf8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:06.456317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c6a6f-4e19-41eb-ae78-ab1a35ab3bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:06.456317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c6a6f-4e19-41eb-ae78-ab1a35ab3bf8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:06.457528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 601/1201] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:26:06.463776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:06.464735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:06.465368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:06.486232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:06.487586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:06.622428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 601/1202] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:26:06.911583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c6a6f-4e19-41eb-ae78-ab1a35ab3bf8 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:07.105272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c6a6f-4e19-41eb-ae78-ab1a35ab3bf8 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:07.165381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:07.166085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:07.166561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:07.167026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:07.188164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:07.190927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:07.217498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:07.219070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:07.220104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:07.220104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:07.220104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:07.220962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 602/1203] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:07.229732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:07.396631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e599dee-d043-47bb-a3ec-18f4698c2579 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:07.445876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:07.456379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:07.457366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:07.457704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:07.458074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:07.458788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:07.459157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:07.460779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eb119d-8ea3-421f-a123-58cffacbf633 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:07.480783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 602/1204] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:07.482611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:07.548090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:07.603222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019c2a9-b9cd-47e2-b861-caa89d4e7f70 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:07.649571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fa7bd6-3f25-48c3-9a1f-123dac39f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:07.650352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa7bd6-3f25-48c3-9a1f-123dac39f370 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:07.650627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa7bd6-3f25-48c3-9a1f-123dac39f370 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:07.650957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fa7bd6-3f25-48c3-9a1f-123dac39f370 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:07.651875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa7bd6-3f25-48c3-9a1f-123dac39f370 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:07.653334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fa7bd6-3f25-48c3-9a1f-123dac39f370 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:07.698308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 603/1205] 10.222.0.91 () {60 vars in 1132 bytes} [Wed Aug 24 03:26:07 2022] POST /identity/v3/users => generated 251 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:26:10.072624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a8938-015b-44ca-8582-1dd29be7f54a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:10.073217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a8938-015b-44ca-8582-1dd29be7f54a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:10.073217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a8938-015b-44ca-8582-1dd29be7f54a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:10.074561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 603/1206] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:10.080446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:10.082772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:10.082772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:10.117073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:10.118359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:10.199012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 604/1207] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:10.528168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a8938-015b-44ca-8582-1dd29be7f54a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:10.756732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a8938-015b-44ca-8582-1dd29be7f54a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:10.813060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:10.813644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:10.813905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:10.814203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:10.835733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:26:10.842580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:10.847048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:10.848223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 604/1208] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:26:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:10.852333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:10.914469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:11.004071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8289985e-0bf2-4d81-90f9-e74e5a4fc810 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:11.060389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:11.061456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:11.061971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:11.062525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:11.063875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:11.064431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:11.067042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:11.081107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 605/1209] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:26:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:11.090425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:11.164292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:11.245042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffc26cf-eff8-48d3-9464-ed39560020db None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:11.286266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:11.287308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:11.287579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:11.287828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:11.301194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:11.302928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:11.319145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:11.320370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:11.321560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:11.321560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:11.321560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:11.324913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 605/1210] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:11.326910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:11.457455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:11.461733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820039e-d053-4e71-9dd8-82faa9c4acc8 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:11.513266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:11.514231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:11.515290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:11.515916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:11.516942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:11.517377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:11.519428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:11.536950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 606/1211] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:11.545456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:11.704992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8facfa50-2a51-45b7-9520-e871b5a00a06 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:11.706742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:11.760881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:11.761544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:11.761919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:11.762204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:11.772383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:11.774053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:26:11.796729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:11.797932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:11.799109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:11.805356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 606/1212] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:11 2022] GET /identity/v3/users/placement => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:11.806458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:11.947046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3beb36-77e5-457b-972d-457efddb0fa9 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:11.999979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:12.000868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:12.001336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:12.001836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:12.003482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:12.004152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:12.007221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:12.040222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 607/1213] 10.222.0.91 () {56 vars in 1143 bytes} [Wed Aug 24 03:26:11 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:12.048156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:12.232205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b84683-1898-4f04-8a1c-91388607f59c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:12.280024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:12.280024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:12.280024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:12.280024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:12.330613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:12.397668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:12.400083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:12.617557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 607/1214] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:26:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:26:12.625412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:12.659239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:12.779183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca79c25-58f7-4580-8e67-7ed9bf6e45a5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:12.835204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:12.836042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:12.836316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:12.836605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:12.848805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:12.850332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:12.868776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:12.878339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:12.880005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:12.880005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:12.880005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:12.880005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 608/1215] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:12.880424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:12.987167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:13.039732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a630c-8884-495f-96fa-c897ba0c2717 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:13.110421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:13.111399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:13.111859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:13.112317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:13.113371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:13.114095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:13.116323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:13.142247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 608/1216] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:13.143940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:13.238456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:13.281752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af4b0b-1ff8-4850-b20a-7547a6eb9b03 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:13.335174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:13.335924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:13.336384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:13.336624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:13.345748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:13.348353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:26:13.370060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:13.371255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:13.372727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:13.373183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 609/1217] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:26:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:13.381281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:13.501481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a9d2b7-a860-4134-8de0-c800683b4f21 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:13.527154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:13.545397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:13.546427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:13.546890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:13.547374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:13.548553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:13.549057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:13.552490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ca1d5b-ae31-4f35-820a-8da183841096 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:13.568702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 609/1218] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:26:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:13.575980 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:13.740857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf0afc3-8737-4f65-9b9c-a06f7d32f8af None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:13.795291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:13.803953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:13.804783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:13.805201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:13.805608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:13.806563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=43c98ae03bf9469bb70ae0ee2e9aecb6) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:13.807033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:13.808696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e618d3-cc1e-4d70-94b5-ccdd466eef2f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:13.819288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 610/1219] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:26:13 2022] GET /identity/v3/role_assignments?user.id=43c98ae03bf9469bb70ae0ee2e9aecb6&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 259 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:15.877127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13219-9bb4-4926-823d-fce3b01461f0 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:15.877728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13219-9bb4-4926-823d-fce3b01461f0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:15.877972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd13219-9bb4-4926-823d-fce3b01461f0 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:15.880231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 610/1220] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:15.886539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:15.886994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:15.887076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:15.912152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:15.913662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:16.013580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 611/1221] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:16.317255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13219-9bb4-4926-823d-fce3b01461f0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:16.499256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd13219-9bb4-4926-823d-fce3b01461f0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:16.556713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:16.557364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:16.557642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:16.557919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:16.569840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:26:16.578849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:16.580887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:16.582798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 611/1222] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:26:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:16.587245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:16.752404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd80e7-2866-40ae-8351-367218e5e0f2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:16.780795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:16.797955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:16.799036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:16.799535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:16.800023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:16.801248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] RBAC: Adding query filter params (name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:16.801730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:16.804299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:16.818630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 612/1223] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:26:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:16.827735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:16.882863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:16.953156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840f308d-2a5d-4e8a-ad0d-26b922acf72c None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:16.994946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:16.995667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:16.996144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:16.996329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:17.009223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:17.010933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:17.026933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:17.028326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:17.029986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:17.029986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:17.029986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:17.029986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 612/1224] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:17.035553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:17.168337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1a37c2-1082-4ab1-9d43-f602f18123aa None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:17.214021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:17.214657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:17.214931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:17.215209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:17.216604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:17.216905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:17.218359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:17.238587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 613/1225] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:17.247702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:17.394666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:17.404890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3601d4-35d4-4b54-9bd9-3e2212a2bab7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:17.447106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:17.447926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:17.448171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:17.448439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:17.461532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:17.463518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:26:17.478753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:17.480599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:17.482079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 613/1226] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:17.489705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:17.543566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:17.608344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bfda1c-5be7-4da2-b6f2-af578d7934dc None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:17.652435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:17.653422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:17.654127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:17.654444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:17.655267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:17.655717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:17.657383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:17.680628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 614/1227] 10.222.0.91 () {56 vars in 1143 bytes} [Wed Aug 24 03:26:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:17.687822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:17.832761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1056e869-86f2-4940-9df0-664d89f2997b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:17.874640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:17.875214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:17.875490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:17.875831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:17.891027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:17.892619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:17.908444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:17.909640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:17.910599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:17.910599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:17.910599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:17.910599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 614/1228] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:17.921162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:18.051340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e185c168-e1ad-40c2-98f7-08218e6b6336 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:18.053902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:18.093361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:18.093361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:18.093361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:18.093361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:18.093956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:18.093956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:18.094738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:18.116781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 615/1229] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:18.118057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:18.129143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:18.240026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e2ff30-9333-4ebc-93f5-dc14e4c03314 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:18.285463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:18.286098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:18.287733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:18.287733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:18.295705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:18.297661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:26:18.311724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:18.312807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:18.314342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 615/1230] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:26:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:18.319627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:18.445987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a0adf-481b-43fd-8d03-fc8e579dd939 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:18.492871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:18.493439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:18.493698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:18.494200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:18.494970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:18.495459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:18.497964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415148eb-ff96-461b-a868-d1a6553e6874 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:18.515911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 616/1231] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:26:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:18.522344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:18.599877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:18.646909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5523471-7897-47a4-b7f6-64215df5a030 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:18.689405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f746423b-17f5-4fca-8e3a-676de6385469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:18.690224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746423b-17f5-4fca-8e3a-676de6385469 None admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:18.690565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746423b-17f5-4fca-8e3a-676de6385469 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:18.690991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f746423b-17f5-4fca-8e3a-676de6385469 None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/43c98ae03bf9469bb70ae0ee2e9aecb6/roles/791aec27d9d243d5a7b1a861db32750e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:18.710829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746423b-17f5-4fca-8e3a-676de6385469 None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=43c98ae03bf9469bb70ae0ee2e9aecb6, role_id=791aec27d9d243d5a7b1a861db32750e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:18.712979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f746423b-17f5-4fca-8e3a-676de6385469 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:18.728305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 616/1232] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:26:18 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/43c98ae03bf9469bb70ae0ee2e9aecb6/roles/791aec27d9d243d5a7b1a861db32750e => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:26:20.859782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d5d4-17df-4538-9f04-f8cf2dbc5d37 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:20.860407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d5d4-17df-4538-9f04-f8cf2dbc5d37 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:20.860407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b919d5d4-17df-4538-9f04-f8cf2dbc5d37 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:20.861658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 617/1233] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:20.868158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:20.868943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:20.869188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:20.894387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:20.895783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:21.011237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 617/1234] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:26:21.341122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d5d4-17df-4538-9f04-f8cf2dbc5d37 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:21.385284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:21.496460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b919d5d4-17df-4538-9f04-f8cf2dbc5d37 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:21.538614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:21.539341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:21.539647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:21.539911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:21.552566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:26:21.557316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:21.562740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:21.563844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 618/1235] 10.222.0.91 () {56 vars in 1102 bytes} [Wed Aug 24 03:26:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:21.567503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:21.657292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:21.754854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa939f0-f418-4be9-9c9c-b7dcda46f010 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:21.800932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:21.801879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:21.802292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:21.802809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:21.803731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:21.804155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:21.805954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:21.819483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 618/1236] 10.222.0.91 () {56 vars in 1103 bytes} [Wed Aug 24 03:26:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:21.825930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:21.943095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:21.966730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4022915-1b0f-4667-872f-77efd032e658 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:22.019900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:22.020537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:22.020787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:22.021060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:22.037309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:22.038852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:22.055270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:22.056526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:22.057865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:22.057865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:22.057865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:22.057865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 619/1237] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:22.066368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:22.237213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9de333-961f-4b66-87b9-f747a76fcdc2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:22.253203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:22.289196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:22.290116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:22.290555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:22.290898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:22.291812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:22.292221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:22.294443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:22.310420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 619/1238] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:22.316249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:22.446836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48d361-009c-4337-a67c-4cfe167492a2 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:22.487321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:22.488230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:22.488697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:22.489157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:22.494317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:22.502986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:22.504927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:26:22.519176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:22.520626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:22.521847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 620/1239] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:22.530503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:22.681251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d7a9da-1d51-4aae-81d2-982429048143 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:22.738221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:22.739134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:22.739686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:22.740237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:22.741548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:22.742229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:22.745094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:22.773242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 620/1240] 10.222.0.91 () {56 vars in 1143 bytes} [Wed Aug 24 03:26:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:22.779884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:22.908739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a10904f-1a8d-4e07-98f2-1bcec75f7a1e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:22.948645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:22.949368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:22.949575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:22.950372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:23.039882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:23.054629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:23.057397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:23.206150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 621/1241] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:26:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:26:23.214440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:23.256768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:23.362861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158f56-8dc5-443d-a0ac-dd051f8eab10 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:23.414375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:23.415045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:23.415299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:23.415575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:23.427300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:23.429557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:23.452374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:23.453903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:23.455390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:23.455390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:23.455390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:23.455390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 621/1242] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:23.460441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:23.583655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:23.591963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c5d56f-af0c-46eb-8a1b-71818c380c4e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:23.636995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:23.638510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:23.638510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:23.639970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:23.639970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:23.639970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:23.641365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:23.657795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 622/1243] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:23.665016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:23.778226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:23.805228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f50a5d7-cd52-4bcd-a4dd-fd4d3e14f12b None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:23.855895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:23.856710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:23.857056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:23.857358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:23.866141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:23.867648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:26:23.881260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:23.883394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:23.885338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:23.887907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 622/1244] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:26:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:23.888873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:24.010740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d76df3-2f5e-44d3-a57e-d2c6978ca464 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:24.057488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:24.058553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:24.059592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:24.060095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:24.061392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:24.061915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:24.064756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44119e98-5d6e-489f-bccf-e9fe76c06ee4 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:24.087799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 623/1245] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:26:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:24.094274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:24.225949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4906cfb4-6e92-4302-8b1d-4eb0f41a19bc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:24.265534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:24.266495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:24.266870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:24.267175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:24.268204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] RBAC: Adding query filter params (role.id=791aec27d9d243d5a7b1a861db32750e, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=43c98ae03bf9469bb70ae0ee2e9aecb6) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:24.268690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:24.270553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869fd6c4-cb78-436a-aa88-549eb95277ee None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:24.278463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 623/1246] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:26:24 2022] GET /identity/v3/role_assignments?user.id=43c98ae03bf9469bb70ae0ee2e9aecb6&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=791aec27d9d243d5a7b1a861db32750e => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:26.425382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519212b0-04ef-4d4a-8976-f77f72184b7c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:26.426235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519212b0-04ef-4d4a-8976-f77f72184b7c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:26.426235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519212b0-04ef-4d4a-8976-f77f72184b7c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:26.427319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 624/1247] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:26:26.433724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:26.434547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:26.434547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:26.442837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:26.463487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:26.464883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:26.568592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 624/1248] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 24 03:26:26.891494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519212b0-04ef-4d4a-8976-f77f72184b7c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:26.920695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:27.058546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519212b0-04ef-4d4a-8976-f77f72184b7c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:27.102042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:27.102684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:27.102967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:27.103809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:27.116033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:26:27.120868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:27.122639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:27.123534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 625/1249] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:26:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:27.132674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:27.285606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d61faa7-83c8-4a08-8975-bdfd1f19c854 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:27.329742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:27.337128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:27.337981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:27.338094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:27.338343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:27.339055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:27.339359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:27.340819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:27.360132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 625/1250] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:26:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:27.361450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:27.472867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:27.509664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bdf3bb-009b-4b86-8374-3be852afe94e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:27.560428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:27.560996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:27.561278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:27.561549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:27.579387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:27.581021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:27.601211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:27.602504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:27.603786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:27.603786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:27.603786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:27.603786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 626/1251] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:27.609573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:27.742541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c24fa4-6c73-43bc-ba09-ecd0a9274ebc None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:27.789761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:27.790441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:27.790716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:27.790996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:27.791737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:27.791987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:27.793950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:27.818622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 626/1252] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:27.825776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:27.955144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5560fe-ce1b-4a2b-834b-2d21a5ba9149 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:27.992758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:28.004426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:28.005132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:28.005473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:28.005993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:28.017269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:28.019045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:26:28.037148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:28.039158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:28.041128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 627/1253] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:28.047645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:28.137988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:28.211598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12434439-4010-4bc9-a4fe-a6045d934276 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:28.268155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:28.269119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:28.269602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:28.270128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:28.271485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:28.271914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:28.274022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:28.303433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 627/1254] 10.222.0.91 () {56 vars in 1143 bytes} [Wed Aug 24 03:26:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:28.311359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:28.475561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384af9ee-2e28-489c-b5d3-0dff8bf5e8b7 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:28.524754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:28.525449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:28.525727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:28.526060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:28.611585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:28.630142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:28.631651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:28.771171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 628/1255] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:26:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:26:28.778165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:28.948823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f860ce-0606-4c5b-a9ae-29db201137c7 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:28.968505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:28.996707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:28.997531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:28.997928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:28.998377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:29.011054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:29.012770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:29.029952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:29.031109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:29.032924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:29.032924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:29.032924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:29.032924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 628/1256] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:29.039878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:29.159526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:29.177283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec0e0d8-4b9d-4731-b097-918cbb54d66f None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:29.237304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:29.238392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:29.238996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:29.239463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:29.241623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:29.242092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:29.244785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:29.267424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 629/1257] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:29.268786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:29.436042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca25e081-e28b-440f-8bd7-bae272f3f76e None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:29.495979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:29.497555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:29.498049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:29.498572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:29.511565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:29.514206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:26:29.535126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:29.537534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:29.539829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 629/1258] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:26:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:29.544435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:29.687916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c4c3e5-7332-4555-bc69-ad96b51d4709 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:29.729590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:29.730227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:29.730542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:29.730818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:29.731554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:29.731863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:29.733313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114083cd-eabf-494c-80da-6870bcde5d83 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:29.746608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 630/1259] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:26:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:29.754352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:29.788050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:29.889340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000a473-f2bf-4e47-8fce-46aa1f11829a None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:29.949421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:29.950309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:29.950664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:29.950934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:29.951676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=43c98ae03bf9469bb70ae0ee2e9aecb6) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:29.951976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:29.953423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a18911-1b05-40a5-99d9-f56d1c53c774 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:29.960213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 630/1260] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:26:29 2022] GET /identity/v3/role_assignments?user.id=43c98ae03bf9469bb70ae0ee2e9aecb6&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:32.077563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbe81-837c-40fb-88a5-12d77e5fed56 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:32.078139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbe81-837c-40fb-88a5-12d77e5fed56 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:32.078139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430bbe81-837c-40fb-88a5-12d77e5fed56 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:32.079474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 631/1261] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:32.085101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:32.085850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:32.085850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:32.105569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:32.106917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:32.190740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 631/1262] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:32.455469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bbe81-837c-40fb-88a5-12d77e5fed56 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:32.473407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:32.569753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430bbe81-837c-40fb-88a5-12d77e5fed56 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:32.604632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:32.605258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:32.605539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:32.605812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:32.615936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:26:32.620579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:32.622459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:32.624258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 632/1263] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:26:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:32.629766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:32.659179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:32.748939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2c328c-9bb2-423e-8f97-adcc3f66bd9b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:32.785991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:32.786878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:32.787155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:32.787426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:32.788083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:32.788383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:32.789747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:32.798656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 632/1264] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:26:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:32.804543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:32.877680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:32.914709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adaaa854-2d1b-4d8f-88b9-d29ff2424673 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:32.951857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:32.952470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:32.952744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:32.953010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:32.964332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:32.967008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:32.982140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:32.983626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:32.984770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:32.984770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:32.984770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:32.984770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 633/1265] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:32.989294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:33.092088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:33.115656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc691777-28ef-4e5c-ae92-f69aa2bdb613 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:33.158299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:33.158921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:33.159199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:33.159473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:33.160117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:33.160417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:33.162044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:33.174057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 633/1266] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:33.180097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:33.285858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55708f66-e909-48b0-b714-ddb49238abfd None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:33.306228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:33.322389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:33.323015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:33.323297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:33.323569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:33.332092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:33.333548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:26:33.344614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:33.346584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:33.348592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 634/1267] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:33 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:33.351935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:33.474740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df676b94-29d6-47c7-a650-d51ce97bab19 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:33.513012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:33.513615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:33.513893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:33.514153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:33.514986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:33.515382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:33.525435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:33.535217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:33.547012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 634/1268] 10.222.0.91 () {56 vars in 1143 bytes} [Wed Aug 24 03:26:33 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:33.554393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:33.687081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bcbdc0-3c24-4189-9bef-02c1485a0a87 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:33.725383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:33.726007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:33.726311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:33.726582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:33.738353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:33.739809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:33.754835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:33.756046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:33.756977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:33.756977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:33.756977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:33.756977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 635/1269] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:33.763464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:33.878018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f75fc-f342-4083-b083-9c65a1fdb214 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:33.913463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:33.914054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:33.914367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:33.914640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:33.915276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:33.915576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:33.916940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:33.928918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 635/1270] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:33.936134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:33.973500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:34.053183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74586007-7d2c-4f49-bf68-de7b92159078 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:34.087461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:34.088232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:34.088539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:34.088897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:34.096904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:34.098737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:26:34.111572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:34.113225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:34.115266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 636/1271] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:26:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:34.118671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:34.188627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:34.232714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afab060-a88b-498e-ba1b-5924369f26f0 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:34.269069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:34.269689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:34.269966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:34.270267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:34.270971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:34.271271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:34.272710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fde841e-28b4-4cdc-b71b-2c5865c400cf None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:34.285713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 636/1272] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:26:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:34.292417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:34.390267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:34.400303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b7bd98-6ff9-4414-b9f6-084cf905c9e1 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:34.435532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7713f6-4f48-4260-bd16-35be2672649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:34.436322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7713f6-4f48-4260-bd16-35be2672649e None admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:34.436599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7713f6-4f48-4260-bd16-35be2672649e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:34.436868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7713f6-4f48-4260-bd16-35be2672649e None admin] PATH_INFO: `/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/43c98ae03bf9469bb70ae0ee2e9aecb6/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:34.438805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7713f6-4f48-4260-bd16-35be2672649e None admin] RBAC: Authorizing `identity:create_grant(project_id=02199521c8284ce2a5b47266bc213ef5, user_id=43c98ae03bf9469bb70ae0ee2e9aecb6, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:34.440352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7713f6-4f48-4260-bd16-35be2672649e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:34.456504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 637/1273] 10.222.0.91 () {58 vars in 1439 bytes} [Wed Aug 24 03:26:34 2022] PUT /identity/v3/projects/02199521c8284ce2a5b47266bc213ef5/users/43c98ae03bf9469bb70ae0ee2e9aecb6/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:26:36.407413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9313-1107-4943-874c-26025ab9d9ed None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:36.408001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9313-1107-4943-874c-26025ab9d9ed None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:36.408001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468c9313-1107-4943-874c-26025ab9d9ed None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:36.409356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 637/1274] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:36.416438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26786420-34da-4871-a3c5-0b504c444849 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:36.416969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26786420-34da-4871-a3c5-0b504c444849 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:36.417063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26786420-34da-4871-a3c5-0b504c444849 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:36.446611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-26786420-34da-4871-a3c5-0b504c444849 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:36.448706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-26786420-34da-4871-a3c5-0b504c444849 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:36.549711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 638/1275] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:36.825438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c9313-1107-4943-874c-26025ab9d9ed None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:36.939287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468c9313-1107-4943-874c-26025ab9d9ed None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:36.961637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:36.989670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:36.990736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:36.991225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:36.991719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:37.006043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 24 03:26:37.010956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:37.012952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:37.014897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 638/1276] 10.222.0.91 () {56 vars in 1096 bytes} [Wed Aug 24 03:26:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:37.019147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26786420-34da-4871-a3c5-0b504c444849 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:37.121761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:37.143679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26786420-34da-4871-a3c5-0b504c444849 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:37.180398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:37.181418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:37.181782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:37.182095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:37.183007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:37.183378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:37.184815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:37.193804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 639/1277] 10.222.0.91 () {56 vars in 1099 bytes} [Wed Aug 24 03:26:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:37.200122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:37.314420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaafc81f-06f6-40d6-b6cf-7690b2607cd4 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:37.349971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:37.350630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:37.350905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:37.351173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:37.363122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:37.364488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:37.378784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:37.379890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:37.380824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:37.380824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:37.380824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:37.380824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 639/1278] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:37.386269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:37.496709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eef30eb-5576-43f1-8bbf-9da109e3a4e5 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:37.537107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:37.537946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:37.538450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:37.538838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:37.539559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:37.539861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:37.541260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:37.554949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 640/1279] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:37.561677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:37.603702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:37.692515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c48817-af0b-43ff-9ee3-a566a3962d3f None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:37.728967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:37.729601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:37.729892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:37.730165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:37.739742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:37.741189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 24 03:26:37.752994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:37.755177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:37.757375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 640/1280] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:37 2022] GET /identity/v3/users/placement => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:37.762571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:37.766693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:37.893477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fb76c0-25d3-4ba3-8b9d-b2542c01bd23 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:37.934227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:37.934850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:37.935132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:37.935422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:37.936182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:37.936494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:37.937962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:37.958217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 641/1281] 10.222.0.91 () {56 vars in 1143 bytes} [Wed Aug 24 03:26:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:37.964026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:38.075353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:38.089741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a84a6b-071e-4ca1-870c-464e7d943ba8 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:38.144018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:38.145133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:38.145637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:38.146115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:38.259654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:38.261057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:38.381496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:38.394274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 641/1282] 10.222.0.91 () {58 vars in 1282 bytes} [Wed Aug 24 03:26:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 24 03:26:38.401357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:38.505126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:38.553233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a148a1-c0d0-440b-b720-961050c0155e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:38.596806 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:38.597742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:38.598257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:38.598536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:38.612659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:38.614353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 24 03:26:38.640677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:38.642336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 24 03:26:38.643906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 24 03:26:38.643906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 24 03:26:38.643906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:38.643906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 642/1283] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:26:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:38.650286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:38.794403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081481cb-62ee-4c6d-a945-e72747228654 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:38.844954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:38.845567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:38.845893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:38.846267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:38.847035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:38.847396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:38.848965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:38.870406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 642/1284] 10.222.0.91 () {56 vars in 1109 bytes} [Wed Aug 24 03:26:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:38.879254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:39.034034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb1f25d-2d2e-4533-aa13-4b3325984b71 None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:39.055203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:39.081761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:39.082603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:39.082983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:39.083363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:39.091988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:39.093477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 24 03:26:39.105804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:39.107689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: ERROR keystone.server.flask.application Aug 24 03:26:39.109177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 643/1285] 10.222.0.91 () {56 vars in 1111 bytes} [Wed Aug 24 03:26:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:39.113583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:39.188006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:39.264037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e380e1-883d-4083-b5f0-8f1edb5f2637 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:39.312284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:39.313266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:39.313731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:39.314214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:39.315455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:39.315983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:39.318297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa13b0-4683-4c20-8a91-dc69f922927a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:39.338124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 643/1286] 10.222.0.91 () {56 vars in 1148 bytes} [Wed Aug 24 03:26:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:39.344694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:39.473377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fc3631-d8b1-41a1-b075-60e25ba2b66e None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:39.534033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:39.535052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:39.535504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:39.535901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:39.537344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] RBAC: Adding query filter params (role.id=4edad00f66404dc7bc651b8ed8896064, scope.project.id=02199521c8284ce2a5b47266bc213ef5, user.id=43c98ae03bf9469bb70ae0ee2e9aecb6) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:39.537836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:39.540273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df2ba93-1247-4e88-8717-31cfb9e4abbe None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:39.550339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 644/1287] 10.222.0.91 () {56 vars in 1374 bytes} [Wed Aug 24 03:26:39 2022] GET /identity/v3/role_assignments?user.id=43c98ae03bf9469bb70ae0ee2e9aecb6&scope.project.id=02199521c8284ce2a5b47266bc213ef5&role.id=4edad00f66404dc7bc651b8ed8896064 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:41.548706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b4eb-e763-4628-b068-fab001067e1f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.549258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b4eb-e763-4628-b068-fab001067e1f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.549681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d031b4eb-e763-4628-b068-fab001067e1f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.551261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 644/1288] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:41.557084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:41.557604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:41.557604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:41.576977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:41.578242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:41.609243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:41.664116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 645/1289] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:41.946173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031b4eb-e763-4628-b068-fab001067e1f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:42.064557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d031b4eb-e763-4628-b068-fab001067e1f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:42.076395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:42.104335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:42.105555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.105830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.106099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.106721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:42.108121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:42.117175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:42.118484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:42.119388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 645/1290] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:26:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 24 03:26:42.125020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:42.226804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fb6d7-43c8-4db6-8ebc-08e02c5323ac None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:42.262482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:42.263092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.263370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.263638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.264272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:42.264577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:42.265948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ace4935-2efd-460f-8f33-881fa1cc96c1 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:42.270956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 646/1291] 10.222.0.91 () {56 vars in 1116 bytes} [Wed Aug 24 03:26:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:26:42.276888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:42.390824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b57511-0e0c-45be-b976-af8bc3b533d2 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:42.429437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:42.430457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:42.430939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:42.431376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] PATH_INFO: `/v3/services` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:42.432497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] RBAC: Adding query filter params (type=placement) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:42.433053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:42.435575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dc7dcd-7d6c-48d1-a4a4-d586557d949b None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:42.443413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 646/1292] 10.222.0.91 () {56 vars in 1116 bytes} [Wed Aug 24 03:26:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:44.390441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53669418-5599-4e60-9c11-96d96ea02999 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:44.391294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53669418-5599-4e60-9c11-96d96ea02999 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:44.391294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53669418-5599-4e60-9c11-96d96ea02999 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:44.392252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 647/1293] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:44.398349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d099b-9426-4ae2-abc2-a9c6669299ca None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:44.399086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d099b-9426-4ae2-abc2-a9c6669299ca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:44.399086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764d099b-9426-4ae2-abc2-a9c6669299ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:44.421214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-764d099b-9426-4ae2-abc2-a9c6669299ca None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:44.422521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-764d099b-9426-4ae2-abc2-a9c6669299ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:44.454612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:44.495756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 647/1294] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:44.775235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53669418-5599-4e60-9c11-96d96ea02999 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:44.808331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:44.890811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53669418-5599-4e60-9c11-96d96ea02999 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:44.929974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b30fb3-6a2a-4337-b0ab-1acc0bbf8931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:44.930673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b30fb3-6a2a-4337-b0ab-1acc0bbf8931 None admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:44.930995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b30fb3-6a2a-4337-b0ab-1acc0bbf8931 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:44.931348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b30fb3-6a2a-4337-b0ab-1acc0bbf8931 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:44.932109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b30fb3-6a2a-4337-b0ab-1acc0bbf8931 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:44.934318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b30fb3-6a2a-4337-b0ab-1acc0bbf8931 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:44.951049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 648/1295] 10.222.0.91 () {60 vars in 1141 bytes} [Wed Aug 24 03:26:44 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:26:46.904386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d22f6-2d26-4b7c-a146-46b7125ffb48 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:46.905040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d22f6-2d26-4b7c-a146-46b7125ffb48 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:46.905040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d22f6-2d26-4b7c-a146-46b7125ffb48 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:46.906390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 648/1296] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:46.917155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:46.917718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:46.917829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:46.937508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:46.938821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:47.031434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 649/1297] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:46 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:47.301394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d22f6-2d26-4b7c-a146-46b7125ffb48 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:47.429666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d22f6-2d26-4b7c-a146-46b7125ffb48 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:47.464069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:47.464691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:47.464962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:47.465225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:47.465795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:47.467176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:47.475671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:47.477318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:47.478364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 649/1298] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:26:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:47.485624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:47.502428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:47.594826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b6ffaa-a138-47ff-905e-95ebc44f39b1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:47.631278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:47.631944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:47.632217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:47.632492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:47.633136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:47.633435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:47.634860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:47.640539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 650/1299] 10.222.0.91 () {56 vars in 1116 bytes} [Wed Aug 24 03:26:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:47.647207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:47.687651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:47.774211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc62b7f-db8a-404d-994c-9d58933051ce None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:47.808353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:47.808949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:47.809218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:47.809480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:47.810229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] RBAC: Adding query filter params (interface=public, service_id=bb6b76cd6f3f4546bb04b72082c92a3e, region_id=RegionOne) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:47.810529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:47.811876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f340631-8472-4652-bc04-1cbda1fa701e None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:47.816857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 650/1300] 10.222.0.91 () {56 vars in 1251 bytes} [Wed Aug 24 03:26:47 2022] GET /identity/v3/endpoints?service_id=bb6b76cd6f3f4546bb04b72082c92a3e&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:47.824306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:47.911386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:47.937846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5951f1-bdf1-440b-a3e1-8e484193026c None admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:47.985575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1449838-5ff3-4862-b334-4f2c18058c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:47.986204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449838-5ff3-4862-b334-4f2c18058c38 None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:47.986490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449838-5ff3-4862-b334-4f2c18058c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:47.986770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1449838-5ff3-4862-b334-4f2c18058c38 None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:47.987372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1449838-5ff3-4862-b334-4f2c18058c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:47.988751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1449838-5ff3-4862-b334-4f2c18058c38 None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:47.995520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 651/1301] 10.222.0.91 () {56 vars in 1087 bytes} [Wed Aug 24 03:26:47 2022] GET /identity/v3/services => generated 2191 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:26:49.915715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a617f2-52fd-4311-a8e7-7ccb8dbfcfb4 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:49.916447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a617f2-52fd-4311-a8e7-7ccb8dbfcfb4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:49.916447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a617f2-52fd-4311-a8e7-7ccb8dbfcfb4 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:49.917621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 651/1302] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:26:49.923024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:49.923492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:49.923645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:49.942308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:26:49.943511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:26:50.025441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 652/1303] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:26:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 24 03:26:50.306271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a617f2-52fd-4311-a8e7-7ccb8dbfcfb4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:50.408467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:50.416489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a617f2-52fd-4311-a8e7-7ccb8dbfcfb4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:50.462845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:50.463483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:50.464289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:50.464560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:50.465144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:50.466536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 24 03:26:50.475438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 24 03:26:50.476771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.get_or_create( Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application with Lock( Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self._enter() Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application return self.creator() Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application created_value = creator( Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: ERROR keystone.server.flask.application Aug 24 03:26:50.478601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 652/1304] 10.222.0.91 () {56 vars in 1117 bytes} [Wed Aug 24 03:26:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 24 03:26:50.484187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:50.569573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:26:50.595780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f5b2e6-c3a8-4631-b1cb-f929eebeaeb3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:50.636490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:50.637154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:50.637474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:50.637823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] PATH_INFO: `/v3/services` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:50.638540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] RBAC: Adding query filter params (name=placement) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:26:50.638877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:50.640399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebd4249-cf9f-43ff-9aaf-8156b761c37b None admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:50.646228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 653/1305] 10.222.0.91 () {56 vars in 1116 bytes} [Wed Aug 24 03:26:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:26:50.654271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:26:50.775748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f10b8c-f2aa-468e-b0ce-fdd6f85aed83 None admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:26:50.816133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63b7ca0-e22f-4342-91b6-9c56296b698a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:26:50.816735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7ca0-e22f-4342-91b6-9c56296b698a None admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:26:50.817012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7ca0-e22f-4342-91b6-9c56296b698a None admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:26:50.817282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b7ca0-e22f-4342-91b6-9c56296b698a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:26:50.817937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b7ca0-e22f-4342-91b6-9c56296b698a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:26:50.819333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63b7ca0-e22f-4342-91b6-9c56296b698a None admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:26:50.853884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 653/1306] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:26:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:27:00.643500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=30 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:27:00.839171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:27:56.743356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538cbc4-f677-4788-9e1a-8305c7f783d0 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:27:56.743804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538cbc4-f677-4788-9e1a-8305c7f783d0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:27:56.744103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9538cbc4-f677-4788-9e1a-8305c7f783d0 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:27:56.745426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 654/1307] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:27:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:27:56.757647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58684d-01b2-4c35-8e8c-32f4f554c5f6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:27:56.760145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58684d-01b2-4c35-8e8c-32f4f554c5f6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:27:56.761309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58684d-01b2-4c35-8e8c-32f4f554c5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:27:56.948642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3a58684d-01b2-4c35-8e8c-32f4f554c5f6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:27:56.949884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3a58684d-01b2-4c35-8e8c-32f4f554c5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:27:57.110363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 654/1308] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:27:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 358 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:27:57.426160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac362be-0d50-495e-9eaf-adaaf8a1929e None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:27:57.426783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac362be-0d50-495e-9eaf-adaaf8a1929e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:27:57.427216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac362be-0d50-495e-9eaf-adaaf8a1929e None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:27:57.429097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 655/1309] 10.222.0.91 () {54 vars in 965 bytes} [Wed Aug 24 03:27:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:27:57.436459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40d4a-420d-4edd-9604-ff416d09bf65 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:27:57.436751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40d4a-420d-4edd-9604-ff416d09bf65 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:27:57.437014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d40d4a-420d-4edd-9604-ff416d09bf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:27:57.529836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-66d40d4a-420d-4edd-9604-ff416d09bf65 None None] MFA Rules not processed for user `5ae7930c17434ae1b6df030b0599e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:27:57.531265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-66d40d4a-420d-4edd-9604-ff416d09bf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:27:57.555537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:27:57.621792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 655/1310] 10.222.0.91 () {58 vars in 1062 bytes} [Wed Aug 24 03:27:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:27:57.631545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac362be-0d50-495e-9eaf-adaaf8a1929e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:27:57.634165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bac362be-0d50-495e-9eaf-adaaf8a1929e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:27:57.770229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac362be-0d50-495e-9eaf-adaaf8a1929e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:27:57.862140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:27:57.863293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:27:57.863729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:27:57.864153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:27:57.866148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:27:57.930154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:27:57.997712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:27:57.999213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97039209-7cf0-46f2-93c5-e1585fd43793 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:27:58.020200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 656/1311] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:00.343490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f08c1-e7e2-40ad-b70e-a41c08543560 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:00.344092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f08c1-e7e2-40ad-b70e-a41c08543560 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:00.344494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723f08c1-e7e2-40ad-b70e-a41c08543560 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:00.347077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 656/1312] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:00.354977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0ab3-4cf0-43c6-820f-19a87c3eb076 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:00.355896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0ab3-4cf0-43c6-820f-19a87c3eb076 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:00.355896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdd0ab3-4cf0-43c6-820f-19a87c3eb076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:00.499131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2cdd0ab3-4cf0-43c6-820f-19a87c3eb076 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:00.500387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2cdd0ab3-4cf0-43c6-820f-19a87c3eb076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:00.613220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 657/1313] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:28:00.898551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f08c1-e7e2-40ad-b70e-a41c08543560 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:00.900259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-723f08c1-e7e2-40ad-b70e-a41c08543560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:01.010115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723f08c1-e7e2-40ad-b70e-a41c08543560 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:01.110648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:01.114368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:01.114995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:01.115573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:01.115838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:01.117591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:01.243275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:01.244721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6ccd1d-0099-4cc8-8927-6683740b763e service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:01.263223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 657/1314] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:03.512634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5628af-fac9-4d73-b95d-d77383cebfa4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:03.513209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5628af-fac9-4d73-b95d-d77383cebfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:03.513209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5628af-fac9-4d73-b95d-d77383cebfa4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:03.514675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 658/1315] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:03.520260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b62494-2914-4a55-a2aa-2dc89110c629 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:03.520760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b62494-2914-4a55-a2aa-2dc89110c629 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:03.520861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b62494-2914-4a55-a2aa-2dc89110c629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:03.590063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-37b62494-2914-4a55-a2aa-2dc89110c629 None None] MFA Rules not processed for user `785cf2fb0f794ab8b0d643d7797ff5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:03.591354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-37b62494-2914-4a55-a2aa-2dc89110c629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:03.647081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:03.692818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 658/1316] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:03 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:03.994357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b621b-b14a-4dab-bbf6-ebf6b48e69ab None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:03.994828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b621b-b14a-4dab-bbf6-ebf6b48e69ab None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:03.995234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b621b-b14a-4dab-bbf6-ebf6b48e69ab None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:03.996774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 659/1317] 10.222.0.91 () {54 vars in 965 bytes} [Wed Aug 24 03:28:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:04.003359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cde21-006b-4836-8585-923804fbb3ce None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:04.003726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cde21-006b-4836-8585-923804fbb3ce None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:04.004070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9cde21-006b-4836-8585-923804fbb3ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:04.082816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5b9cde21-006b-4836-8585-923804fbb3ce None None] MFA Rules not processed for user `5ae7930c17434ae1b6df030b0599e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:04.084027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5b9cde21-006b-4836-8585-923804fbb3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:04.159504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 659/1318] 10.222.0.91 () {58 vars in 1062 bytes} [Wed Aug 24 03:28:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:04.169134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b621b-b14a-4dab-bbf6-ebf6b48e69ab None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:04.170840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-697b621b-b14a-4dab-bbf6-ebf6b48e69ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:04.181316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:04.284066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b621b-b14a-4dab-bbf6-ebf6b48e69ab None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:04.363371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:04.363971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:04.364251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:04.364518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:04.366124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:04.433442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:04.481427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:04.482918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064a9d62-f9c5-4dce-a3ce-b45a018ad923 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:04.501317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 660/1319] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:04 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:07.424117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d27df-84e4-49aa-8167-9dce3fcba801 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:07.424648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d27df-84e4-49aa-8167-9dce3fcba801 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:07.424648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d27df-84e4-49aa-8167-9dce3fcba801 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:07.426011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 660/1320] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:07.432083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797bbb2-1049-4b68-97db-ae3336010110 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:07.432514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797bbb2-1049-4b68-97db-ae3336010110 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:07.432625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5797bbb2-1049-4b68-97db-ae3336010110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:07.498634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5797bbb2-1049-4b68-97db-ae3336010110 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:07.499882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5797bbb2-1049-4b68-97db-ae3336010110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:07.594907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 661/1321] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:07 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:07.884701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d27df-84e4-49aa-8167-9dce3fcba801 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:07.886266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c64d27df-84e4-49aa-8167-9dce3fcba801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:07.938938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:07.993601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d27df-84e4-49aa-8167-9dce3fcba801 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:08.069797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:08.070445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:08.070723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:08.071527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:08.073160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:08.183998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:08.198938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:08.200379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00d281-d0e2-4ce1-8d81-44e31fc27287 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:08.220064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 661/1322] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:07 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:10.796411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87f49-c126-448d-b69e-ff12a84293f4 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:10.796990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87f49-c126-448d-b69e-ff12a84293f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:10.796990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf87f49-c126-448d-b69e-ff12a84293f4 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:10.798363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 662/1323] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:10.803737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25144f4e-6051-42ea-98f0-62d890d318c8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:10.804133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25144f4e-6051-42ea-98f0-62d890d318c8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:10.804284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25144f4e-6051-42ea-98f0-62d890d318c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:10.871240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-25144f4e-6051-42ea-98f0-62d890d318c8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:10.872446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-25144f4e-6051-42ea-98f0-62d890d318c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:10.974277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 662/1324] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:11.257527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf87f49-c126-448d-b69e-ff12a84293f4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:11.259973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5cf87f49-c126-448d-b69e-ff12a84293f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:11.281727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:11.374956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf87f49-c126-448d-b69e-ff12a84293f4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:11.460133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:11.460765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:11.461044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:11.461314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:11.463011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:11.546032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:11.591766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:11.593229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5691151e-5e0b-4aa1-9834-959dfac68fe1 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:11.611100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 663/1325] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:13.592540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474d6d-d489-46da-b1ed-93c823b17caf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:13.593076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474d6d-d489-46da-b1ed-93c823b17caf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:13.593076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14474d6d-d489-46da-b1ed-93c823b17caf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:13.594452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 663/1326] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:13.599958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9578313-69a1-404b-bd09-68bd3852dde5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:13.600397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9578313-69a1-404b-bd09-68bd3852dde5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:13.600482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9578313-69a1-404b-bd09-68bd3852dde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:13.673586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d9578313-69a1-404b-bd09-68bd3852dde5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:13.674888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d9578313-69a1-404b-bd09-68bd3852dde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:13.773799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 664/1327] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:14.060068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474d6d-d489-46da-b1ed-93c823b17caf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:14.061586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-14474d6d-d489-46da-b1ed-93c823b17caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:14.129346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:14.173326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14474d6d-d489-46da-b1ed-93c823b17caf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:14.256987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:14.257705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:14.258049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:14.258418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:14.260181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:14.374464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:14.377060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:14.379377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a64c51-8d49-45b5-8c19-8f7017d24e16 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:14.400353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 664/1328] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:16.416994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59385e89-f631-490f-9252-3ef4eb309a43 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:16.417567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59385e89-f631-490f-9252-3ef4eb309a43 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:16.417567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59385e89-f631-490f-9252-3ef4eb309a43 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:16.419092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 665/1329] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:16.424461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c803d-5750-4d03-892d-7a241b6f5b93 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:16.424997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c803d-5750-4d03-892d-7a241b6f5b93 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:16.424997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c803d-5750-4d03-892d-7a241b6f5b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:16.494017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8f1c803d-5750-4d03-892d-7a241b6f5b93 None None] MFA Rules not processed for user `785cf2fb0f794ab8b0d643d7797ff5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:16.495283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f1c803d-5750-4d03-892d-7a241b6f5b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:16.601790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 665/1330] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:16 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:16.889472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59385e89-f631-490f-9252-3ef4eb309a43 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:16.890951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-59385e89-f631-490f-9252-3ef4eb309a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:16.944387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:17.027097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59385e89-f631-490f-9252-3ef4eb309a43 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:17.102570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:17.103230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:17.103507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:17.103775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:17.105383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:17.187296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:17.221540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:17.223026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dcf6d3-cd0f-44d1-879c-626d5f38f8eb service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:17.244369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 666/1331] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:16 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:19.452852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956143d-7d19-4f97-a611-4f185440f946 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:19.453427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956143d-7d19-4f97-a611-4f185440f946 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:19.453427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9956143d-7d19-4f97-a611-4f185440f946 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:19.454784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 666/1332] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:19.460068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd0fc1-790b-41a3-a3ca-fc2148edff33 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:19.460517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd0fc1-790b-41a3-a3ca-fc2148edff33 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:19.460634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd0fc1-790b-41a3-a3ca-fc2148edff33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:19.528093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ddcd0fc1-790b-41a3-a3ca-fc2148edff33 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:19.529321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ddcd0fc1-790b-41a3-a3ca-fc2148edff33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:19.625346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 667/1333] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:19.907884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956143d-7d19-4f97-a611-4f185440f946 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:19.909439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9956143d-7d19-4f97-a611-4f185440f946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:19.978808 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:20.013456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9956143d-7d19-4f97-a611-4f185440f946 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:20.087447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:20.088051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:20.088324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:20.088587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:20.090200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:20.200063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:20.201387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af451df-483e-4f68-a0a8-acb13c649de9 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:20.211324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:20.220958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 667/1334] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:22.244061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f67a8-f70d-4d42-84f7-459b0a40ed40 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:22.244714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f67a8-f70d-4d42-84f7-459b0a40ed40 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:22.244714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387f67a8-f70d-4d42-84f7-459b0a40ed40 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:22.245995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 668/1335] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:22.256158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d7dd-d9a6-4e82-9a7c-bb0edcb0e11e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:22.256722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d7dd-d9a6-4e82-9a7c-bb0edcb0e11e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:22.257110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4875d7dd-d9a6-4e82-9a7c-bb0edcb0e11e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:22.348178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4875d7dd-d9a6-4e82-9a7c-bb0edcb0e11e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:22.349460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4875d7dd-d9a6-4e82-9a7c-bb0edcb0e11e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:22.470760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 668/1336] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:22.807459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f67a8-f70d-4d42-84f7-459b0a40ed40 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:22.810375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-387f67a8-f70d-4d42-84f7-459b0a40ed40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:22.876423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:22.998819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387f67a8-f70d-4d42-84f7-459b0a40ed40 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:23.128353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:23.129332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:23.129810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:23.130177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:23.132887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:23.278328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:23.313800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:23.316603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5fc70ff-86b4-4ad4-a84a-37e48be9f5f2 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:23.353433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 669/1337] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 547 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:25.711809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f522a5f-d0d4-4697-b524-0e0258186a0f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:25.712418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f522a5f-d0d4-4697-b524-0e0258186a0f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:25.712545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f522a5f-d0d4-4697-b524-0e0258186a0f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:25.714034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 669/1338] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:25.720742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968725c-a454-4c20-b65d-063abe3b35c0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:25.721207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968725c-a454-4c20-b65d-063abe3b35c0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:25.721404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5968725c-a454-4c20-b65d-063abe3b35c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:25.798843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5968725c-a454-4c20-b65d-063abe3b35c0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:25.800641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5968725c-a454-4c20-b65d-063abe3b35c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:25.919745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 670/1339] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:28:26.206753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f522a5f-d0d4-4697-b524-0e0258186a0f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:26.208309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0f522a5f-d0d4-4697-b524-0e0258186a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:26.285508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:26.314543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f522a5f-d0d4-4697-b524-0e0258186a0f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:26.388518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:26.389039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:26.389314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:26.390025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:26.391185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:26.505746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:26.507213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f37163-2478-4ea0-b2da-f08d7ae5e66a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:26.525292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 670/1340] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:26 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:30.475172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd693ff-ae9b-4fe0-9872-901589dd963d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:30.475893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd693ff-ae9b-4fe0-9872-901589dd963d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:30.475893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd693ff-ae9b-4fe0-9872-901589dd963d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:30.477103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 671/1341] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:30.482949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3f63-ff03-4ad2-875e-cbca6d6c3ee5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:30.484350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3f63-ff03-4ad2-875e-cbca6d6c3ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:30.485150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60da3f63-ff03-4ad2-875e-cbca6d6c3ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:30.490650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:30.573101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-60da3f63-ff03-4ad2-875e-cbca6d6c3ee5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:30.574367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-60da3f63-ff03-4ad2-875e-cbca6d6c3ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:30.684617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 671/1342] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:30.970374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd693ff-ae9b-4fe0-9872-901589dd963d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:30.971917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4bd693ff-ae9b-4fe0-9872-901589dd963d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:31.026138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:31.090109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd693ff-ae9b-4fe0-9872-901589dd963d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:31.174020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:31.174656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:31.174940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:31.175214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:31.176830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:31.279062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:31.300929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:31.302467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337a4677-bbdf-48d0-bf19-40c59855f704 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:31.322238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 672/1343] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:33.964631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56132e02-e6db-4709-a899-605427810d96 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:33.965177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56132e02-e6db-4709-a899-605427810d96 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:33.965177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56132e02-e6db-4709-a899-605427810d96 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:33.966534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 672/1344] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:33.972556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9639f-3d3e-43cd-b597-c08287ff6134 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:33.973134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9639f-3d3e-43cd-b597-c08287ff6134 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:33.973134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e9639f-3d3e-43cd-b597-c08287ff6134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:34.043770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-92e9639f-3d3e-43cd-b597-c08287ff6134 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:34.044997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-92e9639f-3d3e-43cd-b597-c08287ff6134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:34.147324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 673/1345] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:34.432252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56132e02-e6db-4709-a899-605427810d96 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:34.433806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-56132e02-e6db-4709-a899-605427810d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:34.526157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:34.544972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56132e02-e6db-4709-a899-605427810d96 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:34.635977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:34.636795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:34.637065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:34.637279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:34.638998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:34.772082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:34.773461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61fb4eb-e6c7-4ac0-8a10-652d58003037 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:34.795634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 673/1346] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:34 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:38.823577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d39f5f-6ddc-494a-8abb-6b311bcd5d1d None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:38.824141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d39f5f-6ddc-494a-8abb-6b311bcd5d1d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:38.824141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d39f5f-6ddc-494a-8abb-6b311bcd5d1d None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:38.825465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 674/1347] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:38.831188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aaccc-c99d-4a3f-8ffd-96a36c01ac9e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:38.831689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aaccc-c99d-4a3f-8ffd-96a36c01ac9e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:38.831779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378aaccc-c99d-4a3f-8ffd-96a36c01ac9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:38.854766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:38.908373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-378aaccc-c99d-4a3f-8ffd-96a36c01ac9e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:38.909597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-378aaccc-c99d-4a3f-8ffd-96a36c01ac9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:39.020162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 674/1348] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:39.304034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d39f5f-6ddc-494a-8abb-6b311bcd5d1d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:39.305613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a7d39f5f-6ddc-494a-8abb-6b311bcd5d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:39.367467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:39.425422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d39f5f-6ddc-494a-8abb-6b311bcd5d1d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:39.511799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:39.512412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:39.512682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:39.512946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:39.514646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:39.622564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:39.626557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:39.628046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b71e2-eb0a-4dfb-a900-a1120310c73b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:39.646871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 675/1349] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:44.700952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5638b40-23f7-421f-baa6-525827bd4258 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:44.704231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5638b40-23f7-421f-baa6-525827bd4258 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:44.704231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5638b40-23f7-421f-baa6-525827bd4258 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:44.707151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 675/1350] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:28:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:44.720401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c13145-777e-4f59-892a-165c4fa50715 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:44.720401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c13145-777e-4f59-892a-165c4fa50715 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:44.720929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c13145-777e-4f59-892a-165c4fa50715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:44.828569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-18c13145-777e-4f59-892a-165c4fa50715 None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:44.830034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-18c13145-777e-4f59-892a-165c4fa50715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:44.982940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 676/1351] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:28:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:49.018412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=59 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:49.485583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1adead-58d3-4e48-90ec-e9d3e58d33a8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:49.485934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1adead-58d3-4e48-90ec-e9d3e58d33a8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:49.486297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1adead-58d3-4e48-90ec-e9d3e58d33a8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:49.487688 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 676/1352] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:28:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:49.493274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e8e8-3630-469a-ab98-855fe9b4c31f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:49.493831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e8e8-3630-469a-ab98-855fe9b4c31f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:49.493831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff9e8e8-3630-469a-ab98-855fe9b4c31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:49.563028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:49.588291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1ff9e8e8-3630-469a-ab98-855fe9b4c31f None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:49.589542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1ff9e8e8-3630-469a-ab98-855fe9b4c31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:49.722456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 677/1353] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:28:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:51.151966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dadc4-8979-4c6a-9784-75310d319437 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:51.152505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dadc4-8979-4c6a-9784-75310d319437 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:51.152696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584dadc4-8979-4c6a-9784-75310d319437 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:51.154059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 677/1354] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:51.160039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d62048-36de-4d5c-99ad-ff99c0578c6f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:51.160496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d62048-36de-4d5c-99ad-ff99c0578c6f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:51.160603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d62048-36de-4d5c-99ad-ff99c0578c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:51.239782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e1d62048-36de-4d5c-99ad-ff99c0578c6f None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:51.240994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e1d62048-36de-4d5c-99ad-ff99c0578c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:51.284173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:51.351231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 678/1355] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:28:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:54.634094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7e90-08c9-45f5-9aa9-f0208b519d8e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:54.634664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7e90-08c9-45f5-9aa9-f0208b519d8e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:54.634664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7e90-08c9-45f5-9aa9-f0208b519d8e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:54.635944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 678/1356] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:54.641967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba6b10-0d52-4f4d-8c46-88270e7c2d61 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:54.642449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba6b10-0d52-4f4d-8c46-88270e7c2d61 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:54.642696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ba6b10-0d52-4f4d-8c46-88270e7c2d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:54.724237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-27ba6b10-0d52-4f4d-8c46-88270e7c2d61 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:54.725549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-27ba6b10-0d52-4f4d-8c46-88270e7c2d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:54.865233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 679/1357] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:28:55.182533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5fb54-39e3-487e-9dc8-d3a0a7cbb310 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:55.182811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5fb54-39e3-487e-9dc8-d3a0a7cbb310 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:55.183336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb5fb54-39e3-487e-9dc8-d3a0a7cbb310 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:55.184678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 679/1358] 10.222.0.91 () {54 vars in 962 bytes} [Wed Aug 24 03:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:55.194389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4b3f-b188-4c27-b93b-d1b6e5e1a6a7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:55.194726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4b3f-b188-4c27-b93b-d1b6e5e1a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:55.196322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4b3f-b188-4c27-b93b-d1b6e5e1a6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:55.229484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:55.317515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-211a4b3f-b188-4c27-b93b-d1b6e5e1a6a7 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:55.318655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-211a4b3f-b188-4c27-b93b-d1b6e5e1a6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:55.456395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 680/1359] 10.222.0.91 () {58 vars in 1059 bytes} [Wed Aug 24 03:28:55 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:55.466647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5fb54-39e3-487e-9dc8-d3a0a7cbb310 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:55.468406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fdb5fb54-39e3-487e-9dc8-d3a0a7cbb310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:55.638788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb5fb54-39e3-487e-9dc8-d3a0a7cbb310 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:55.736009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:55.791882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:55.792500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:55.792816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:55.793041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:55.794699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:55.891385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:55.892891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc50e7d8-5116-4568-b259-cdd895dd33bb service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:55.912706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 680/1360] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:28:58.077747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d487643-347d-400f-b736-71cca9ff2e8c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:58.078461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d487643-347d-400f-b736-71cca9ff2e8c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:58.078461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d487643-347d-400f-b736-71cca9ff2e8c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:58.079714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 681/1361] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:28:58.085678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4513-d89f-45ea-8f3c-694f3bb0334f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:58.085678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4513-d89f-45ea-8f3c-694f3bb0334f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:58.085678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cb4513-d89f-45ea-8f3c-694f3bb0334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:58.159678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-13cb4513-d89f-45ea-8f3c-694f3bb0334f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:28:58.160964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-13cb4513-d89f-45ea-8f3c-694f3bb0334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:58.204186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:58.274270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 681/1362] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:28:58.570118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d487643-347d-400f-b736-71cca9ff2e8c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:28:58.572409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d487643-347d-400f-b736-71cca9ff2e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:58.684418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:58.800312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d487643-347d-400f-b736-71cca9ff2e8c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:28:58.942903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:28:58.943650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:28:58.944045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:28:58.944333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:28:58.946318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:28:58.979965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:28:59.062978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:28:59.065169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b078a9-1366-4713-823c-32caebd8c437 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:28:59.093243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 682/1363] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:01.361673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37dbc-a0a0-4836-b241-8e63d1ceaea3 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:01.362228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37dbc-a0a0-4836-b241-8e63d1ceaea3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:01.362940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b37dbc-a0a0-4836-b241-8e63d1ceaea3 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:01.365650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 682/1364] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:01.372137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59830b-ec56-48c5-9971-6327c9c7eb30 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:01.372633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59830b-ec56-48c5-9971-6327c9c7eb30 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:01.372745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c59830b-ec56-48c5-9971-6327c9c7eb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:01.441591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9c59830b-ec56-48c5-9971-6327c9c7eb30 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:01.442842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9c59830b-ec56-48c5-9971-6327c9c7eb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:01.541949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:01.546255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 683/1365] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:01.835303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececc1c4-9ab1-4e83-8729-5395ff4ee62e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:01.838014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececc1c4-9ab1-4e83-8729-5395ff4ee62e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:01.839632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececc1c4-9ab1-4e83-8729-5395ff4ee62e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:01.841378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 683/1366] 10.222.0.91 () {54 vars in 962 bytes} [Wed Aug 24 03:29:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:01.847244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61084357-c967-49df-8ed8-6a038b3a9296 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:01.847533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61084357-c967-49df-8ed8-6a038b3a9296 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:01.847807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61084357-c967-49df-8ed8-6a038b3a9296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:01.928069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-61084357-c967-49df-8ed8-6a038b3a9296 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:01.929278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-61084357-c967-49df-8ed8-6a038b3a9296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:02.055159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 684/1367] 10.222.0.91 () {58 vars in 1059 bytes} [Wed Aug 24 03:29:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:02.063710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececc1c4-9ab1-4e83-8729-5395ff4ee62e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:02.065462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ececc1c4-9ab1-4e83-8729-5395ff4ee62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:02.233471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ececc1c4-9ab1-4e83-8729-5395ff4ee62e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:02.258717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:02.358972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:02.359636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:02.359852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:02.360117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:02.361759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:02.457255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:02.458797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30c5f4-3d6b-471d-951b-e8294422bd8c service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:02.479961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 684/1368] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:29:03.100422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56800949-91f7-4f88-b5b8-608b4ccecaf9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:03.100949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56800949-91f7-4f88-b5b8-608b4ccecaf9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:03.100949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56800949-91f7-4f88-b5b8-608b4ccecaf9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:03.106954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 685/1369] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:03.109188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ce557-e699-4157-943c-f6ddba285ddc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:03.109490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ce557-e699-4157-943c-f6ddba285ddc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:03.109730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5ce557-e699-4157-943c-f6ddba285ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:03.143082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:03.181175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-eb5ce557-e699-4157-943c-f6ddba285ddc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:03.182414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eb5ce557-e699-4157-943c-f6ddba285ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:03.288151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 685/1370] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:03.879093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e4667-3dd5-4aca-8025-eaaa6b55d1ab None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:03.879093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e4667-3dd5-4aca-8025-eaaa6b55d1ab None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:03.879093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4e4667-3dd5-4aca-8025-eaaa6b55d1ab None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:03.881971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 686/1371] 10.222.0.91 () {54 vars in 964 bytes} [Wed Aug 24 03:29:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:03.887343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e0ad0-5cc4-4db8-8f29-64c623cc0f67 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:03.887973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e0ad0-5cc4-4db8-8f29-64c623cc0f67 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:03.887973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47e0ad0-5cc4-4db8-8f29-64c623cc0f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:03.971389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e47e0ad0-5cc4-4db8-8f29-64c623cc0f67 None None] MFA Rules not processed for user `408b15d1a55a439c93a2f879c0d1cc31`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:03.972989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e47e0ad0-5cc4-4db8-8f29-64c623cc0f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:04.032349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:04.069328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 686/1372] 10.222.0.91 () {58 vars in 1061 bytes} [Wed Aug 24 03:29:03 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:04.077476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e4667-3dd5-4aca-8025-eaaa6b55d1ab None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:04.079032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2b4e4667-3dd5-4aca-8025-eaaa6b55d1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:04.135956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:04.197589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4e4667-3dd5-4aca-8025-eaaa6b55d1ab None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:04.279773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:04.280413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:04.280676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:04.280940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:04.282574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:04.431420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:04.447895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:04.449619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24952a-d122-43db-bb6f-1a31d3d24a93 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:04.475339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 687/1373] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:04.508933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4541a-e196-4f7a-987e-1b68fc3f1602 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:04.508933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4541a-e196-4f7a-987e-1b68fc3f1602 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:04.508933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb4541a-e196-4f7a-987e-1b68fc3f1602 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:04.515141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb838cb-b206-4e6b-8db8-b0bb11542fbf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:04.515141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb838cb-b206-4e6b-8db8-b0bb11542fbf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:04.515141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb838cb-b206-4e6b-8db8-b0bb11542fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:04.516296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 687/1374] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:04.583757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3eb838cb-b206-4e6b-8db8-b0bb11542fbf None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:04.585020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3eb838cb-b206-4e6b-8db8-b0bb11542fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:04.690479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 688/1375] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:29:04.984169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4541a-e196-4f7a-987e-1b68fc3f1602 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:04.985707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cfb4541a-e196-4f7a-987e-1b68fc3f1602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:05.140481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb4541a-e196-4f7a-987e-1b68fc3f1602 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:05.194087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:05.256602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:05.257127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:05.257404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:05.257675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:05.259877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:05.355774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:05.357384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad696e3-74ce-45bc-88e2-7461df800321 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:05.376187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 688/1376] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:07.335973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c930-f8bc-4a7c-9d76-7cce289c7256 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:07.336582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c930-f8bc-4a7c-9d76-7cce289c7256 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:07.336582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b1c930-f8bc-4a7c-9d76-7cce289c7256 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:07.338535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 689/1377] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:07.343606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972460d1-3533-491f-a316-943cc230f02a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:07.344095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972460d1-3533-491f-a316-943cc230f02a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:07.344095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972460d1-3533-491f-a316-943cc230f02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:07.423595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-972460d1-3533-491f-a316-943cc230f02a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:07.424866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-972460d1-3533-491f-a316-943cc230f02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:07.432689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:07.595772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 689/1378] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:07.890676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c930-f8bc-4a7c-9d76-7cce289c7256 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:07.892238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-90b1c930-f8bc-4a7c-9d76-7cce289c7256 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:07.973364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:08.101254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b1c930-f8bc-4a7c-9d76-7cce289c7256 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:08.244113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:08.244893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:08.245271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:08.245646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:08.247439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:08.277209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:08.371742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:08.374654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f03884f-8f30-4d06-8ca9-69f7ace4ade1 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:08.395916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 690/1379] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:10.378388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f60aa-3d69-47d2-89e5-b5cd0fd17f42 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:10.378854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f60aa-3d69-47d2-89e5-b5cd0fd17f42 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:10.378936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f60aa-3d69-47d2-89e5-b5cd0fd17f42 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:10.380452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 690/1380] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:10.385588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf885a-a0ee-44ef-a80c-0f57bc606eef None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:10.385985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf885a-a0ee-44ef-a80c-0f57bc606eef None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:10.386166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbf885a-a0ee-44ef-a80c-0f57bc606eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:10.464217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cfbf885a-a0ee-44ef-a80c-0f57bc606eef None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:10.465612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cfbf885a-a0ee-44ef-a80c-0f57bc606eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:10.562396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:10.582565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 691/1381] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:10.863596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f60aa-3d69-47d2-89e5-b5cd0fd17f42 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:10.865164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-390f60aa-3d69-47d2-89e5-b5cd0fd17f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:11.025142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:11.037819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f60aa-3d69-47d2-89e5-b5cd0fd17f42 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:11.173807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:11.174445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:11.174720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:11.174985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:11.176613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:11.281242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:11.282906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add4a3f-b706-4885-822a-a58a239a6f32 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:11.299091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:11.308756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 691/1382] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:13.300415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e74f1-04eb-4ab0-9523-e097095756cb None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.301307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e74f1-04eb-4ab0-9523-e097095756cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.301801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9e74f1-04eb-4ab0-9523-e097095756cb None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.303199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 692/1383] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:13.309784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaf473-29d1-4c89-8824-8392bcf99516 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:13.310447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaf473-29d1-4c89-8824-8392bcf99516 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:13.310447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecaf473-29d1-4c89-8824-8392bcf99516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:13.389166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-eecaf473-29d1-4c89-8824-8392bcf99516 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:13.390665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-eecaf473-29d1-4c89-8824-8392bcf99516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:13.502126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 692/1384] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:29:13.791299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e74f1-04eb-4ab0-9523-e097095756cb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:13.792874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4b9e74f1-04eb-4ab0-9523-e097095756cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:13.972288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9e74f1-04eb-4ab0-9523-e097095756cb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:14.056696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:14.121763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:14.122535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:14.122856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:14.123183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:14.125247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:14.244149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:14.245808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fcddf9-e12d-44f6-9385-3ee8e0a48390 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:14.271228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 693/1385] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:16.319206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d833d-0575-47ea-8eeb-1a3a71004f2b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:16.320419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d833d-0575-47ea-8eeb-1a3a71004f2b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:16.320419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646d833d-0575-47ea-8eeb-1a3a71004f2b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:16.322244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 693/1386] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:16.328853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468850de-a594-4778-b7a0-ff3e9b8e99f1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:16.329739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468850de-a594-4778-b7a0-ff3e9b8e99f1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:16.329739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468850de-a594-4778-b7a0-ff3e9b8e99f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:16.476581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-468850de-a594-4778-b7a0-ff3e9b8e99f1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:16.478915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-468850de-a594-4778-b7a0-ff3e9b8e99f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:16.532219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:16.623884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 694/1387] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:16.989542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d833d-0575-47ea-8eeb-1a3a71004f2b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:16.991190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-646d833d-0575-47ea-8eeb-1a3a71004f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:17.095301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:17.188702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646d833d-0575-47ea-8eeb-1a3a71004f2b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:17.351718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:17.353593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:17.353593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:17.353593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:17.355114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:17.396871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:17.463742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:17.465528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77547e9-2fce-4ace-9e4f-a36acca5d4b1 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:17.491638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 694/1388] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:19.512286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d3a7a-f155-43b3-9e06-2636d4e948c1 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.512890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d3a7a-f155-43b3-9e06-2636d4e948c1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.512890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d3a7a-f155-43b3-9e06-2636d4e948c1 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.514202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 695/1389] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:19.519735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.520318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.520318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.589742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:19.590982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:19.624862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120455-854f-4e6d-a5ee-54470f9746e5 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.625145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120455-854f-4e6d-a5ee-54470f9746e5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.625422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1120455-854f-4e6d-a5ee-54470f9746e5 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.626977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 696/1390] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:19.631639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05f7c-e1b3-444d-bc44-d2f0d5115e76 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:19.631921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05f7c-e1b3-444d-bc44-d2f0d5115e76 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:19.632193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c05f7c-e1b3-444d-bc44-d2f0d5115e76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:19.692009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 695/1391] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:29:19.704101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-15c05f7c-e1b3-444d-bc44-d2f0d5115e76 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:19.705329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-15c05f7c-e1b3-444d-bc44-d2f0d5115e76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:19.803486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 697/1392] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:29:19.983495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:19.984698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:19.998847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:20.103499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673f38c-5b81-4d27-82d2-4664bde8f010 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:20.104220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673f38c-5b81-4d27-82d2-4664bde8f010 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:20.104356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1450c-fa60-4765-a42d-422d7fb56f09 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:20.104828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3673f38c-5b81-4d27-82d2-4664bde8f010 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:20.106714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 698/1393] 10.222.0.91 () {54 vars in 967 bytes} [Wed Aug 24 03:29:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:20.113844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:20.114294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:20.114575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:20.132326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:20.195434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] MFA Rules not processed for user `7d0eef76ae53400c83c391351dc75c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:20.197385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:20.228808 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:20.229428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:20.229710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:20.230021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:20.231659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:20.240800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:20.283059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 699/1394] 10.222.0.91 () {58 vars in 1064 bytes} [Wed Aug 24 03:29:20 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:29:20.290981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:20.298692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc494d0a-7283-495a-a0ba-021344556bb2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:20.328641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:20.330195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a09e79-c9b6-4a2c-a6e9-e246d077b053 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:20.349219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 696/1395] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:20.377291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:20.377909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:20.378219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:20.378491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:20.380121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:20.418996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:20.516521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:20.518015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaa9a55-1d66-4696-9e04-dbf1f3ca5978 service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:20.538130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 700/1396] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:29:21.278643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef331b-926a-4e49-b0fb-c8ddde4826a1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.279128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef331b-926a-4e49-b0fb-c8ddde4826a1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.279128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef331b-926a-4e49-b0fb-c8ddde4826a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.350630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-59ef331b-926a-4e49-b0fb-c8ddde4826a1 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:21.351831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-59ef331b-926a-4e49-b0fb-c8ddde4826a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:21.440433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:21.444133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 697/1397] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:29:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:29:21.488597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e804-4c50-4a67-bd66-27dc4d226624 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.488908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e804-4c50-4a67-bd66-27dc4d226624 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.489184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20e804-4c50-4a67-bd66-27dc4d226624 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.490560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 701/1398] 10.222.0.91 () {54 vars in 943 bytes} [Wed Aug 24 03:29:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:21.497317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83ca0-d872-4b5a-a618-1a5ea587515f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.497668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83ca0-d872-4b5a-a618-1a5ea587515f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.497944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc83ca0-d872-4b5a-a618-1a5ea587515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.598481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dbc83ca0-d872-4b5a-a618-1a5ea587515f None None] MFA Rules not processed for user `c97977ebdd6f4f6da887c8a7215bda69`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:21.599629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dbc83ca0-d872-4b5a-a618-1a5ea587515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:21.717143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 698/1399] 10.222.0.91 () {58 vars in 1040 bytes} [Wed Aug 24 03:29:21 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:21.726698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e804-4c50-4a67-bd66-27dc4d226624 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:21.728246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-aa20e804-4c50-4a67-bd66-27dc4d226624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:21.868501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:21.871569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa20e804-4c50-4a67-bd66-27dc4d226624 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:21.975989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:21.976599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:21.976883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:21.977155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:21.978840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:22.065368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:22.067032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e2e9c6-c5f2-4339-a9ea-d7184fe37c93 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:22.075601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 702/1400] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:29:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:29:22.329397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce86d-b778-4200-b90b-8d46164292cd None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:22.329979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce86d-b778-4200-b90b-8d46164292cd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:22.329979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24ce86d-b778-4200-b90b-8d46164292cd None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:22.335650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 699/1401] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:22.338156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63a451-250a-4778-abdd-6978b391e34d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:22.338797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63a451-250a-4778-abdd-6978b391e34d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:22.339085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d63a451-250a-4778-abdd-6978b391e34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:22.384967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:22.419269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0d63a451-250a-4778-abdd-6978b391e34d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:22.420522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0d63a451-250a-4778-abdd-6978b391e34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:22.532455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 703/1402] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:22.823030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ce86d-b778-4200-b90b-8d46164292cd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:22.824636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f24ce86d-b778-4200-b90b-8d46164292cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:22.892629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:22.997271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24ce86d-b778-4200-b90b-8d46164292cd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:23.142512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:23.143130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:23.143409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:23.143679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:23.145318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:23.165941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:23.250850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:23.252653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1207793e-549e-4cb9-b3b6-4149a77f1510 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:23.273291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 700/1403] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:25.254427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d514-4fea-4119-9612-e3d240b44463 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:25.254986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d514-4fea-4119-9612-e3d240b44463 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:25.254986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d82d514-4fea-4119-9612-e3d240b44463 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:25.256451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 704/1404] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:25.265877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2436-69e6-4b26-92ea-b12b322ab8d0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:25.266457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2436-69e6-4b26-92ea-b12b322ab8d0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:25.266457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2436-69e6-4b26-92ea-b12b322ab8d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:25.343899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e7fb2436-69e6-4b26-92ea-b12b322ab8d0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:25.345178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e7fb2436-69e6-4b26-92ea-b12b322ab8d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:25.459036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:25.467264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 701/1405] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:25.814869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d514-4fea-4119-9612-e3d240b44463 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:25.816409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5d82d514-4fea-4119-9612-e3d240b44463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:25.942418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:26.007060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d82d514-4fea-4119-9612-e3d240b44463 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:26.134014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:26.134654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:26.134940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:26.135209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:26.136848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:26.223647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:26.244618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:26.246252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222ac0b-0605-4bc4-b497-e51040b28a37 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:26.270363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 705/1406] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:28.252511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f80e-56d3-4895-9d94-8456f582784b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.253418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f80e-56d3-4895-9d94-8456f582784b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.253418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a011f80e-56d3-4895-9d94-8456f582784b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.254708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 702/1407] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:28.260602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993012be-e04a-48dd-ada2-18649e19647f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:28.261080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993012be-e04a-48dd-ada2-18649e19647f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:28.261178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993012be-e04a-48dd-ada2-18649e19647f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:28.342250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-993012be-e04a-48dd-ada2-18649e19647f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:28.344045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-993012be-e04a-48dd-ada2-18649e19647f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:28.476416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 706/1408] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:28.782169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011f80e-56d3-4895-9d94-8456f582784b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:28.784136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a011f80e-56d3-4895-9d94-8456f582784b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:28.986894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a011f80e-56d3-4895-9d94-8456f582784b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:29.108302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:29.171154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:29.172088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:29.172367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:29.172641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:29.174449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:29.290092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:29.291979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301898d2-3c00-43ce-93fd-4698865dc62a service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:29.312016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 703/1409] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 530 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:38.474725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=79 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:39.309127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=64 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:44.356040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4751-c509-4956-8c1b-249cd74a1aaa None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.356616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4751-c509-4956-8c1b-249cd74a1aaa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.356616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11c4751-c509-4956-8c1b-249cd74a1aaa None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.357905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 707/1410] 10.222.0.91 () {54 vars in 918 bytes} [Wed Aug 24 03:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:44.363251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ecaa-b413-45ee-9210-9a6377ff6304 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.363676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ecaa-b413-45ee-9210-9a6377ff6304 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.363798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9ecaa-b413-45ee-9210-9a6377ff6304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.439838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-76e9ecaa-b413-45ee-9210-9a6377ff6304 None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:44.441040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-76e9ecaa-b413-45ee-9210-9a6377ff6304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:44.551041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 704/1411] 10.222.0.91 () {58 vars in 1015 bytes} [Wed Aug 24 03:29:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:44.565956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1901b-214d-42b7-80c7-34dd92914114 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.566275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1901b-214d-42b7-80c7-34dd92914114 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.566554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e1901b-214d-42b7-80c7-34dd92914114 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.567873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 708/1412] 10.222.0.91 () {54 vars in 918 bytes} [Wed Aug 24 03:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:44.573079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583e7a-20f5-49e8-83f6-c3a16b5e6862 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:44.573361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583e7a-20f5-49e8-83f6-c3a16b5e6862 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:44.573627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8583e7a-20f5-49e8-83f6-c3a16b5e6862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:44.643761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:44.646950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c8583e7a-20f5-49e8-83f6-c3a16b5e6862 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:44.648683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c8583e7a-20f5-49e8-83f6-c3a16b5e6862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:44.762793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 705/1413] 10.222.0.91 () {58 vars in 1015 bytes} [Wed Aug 24 03:29:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:47.157076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7f7c-c58e-46bf-9369-8ef9888e637b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.158599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7f7c-c58e-46bf-9369-8ef9888e637b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.158599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a7f7c-c58e-46bf-9369-8ef9888e637b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.158960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 709/1414] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:29:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:47.164482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6564a-0707-4e42-a350-5ec42e4cd315 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.164941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6564a-0707-4e42-a350-5ec42e4cd315 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.165039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6564a-0707-4e42-a350-5ec42e4cd315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.231493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-79c6564a-0707-4e42-a350-5ec42e4cd315 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:47.232800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-79c6564a-0707-4e42-a350-5ec42e4cd315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:47.288852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.331467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 706/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:47.624991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a7f7c-c58e-46bf-9369-8ef9888e637b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:47.626725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-795a7f7c-c58e-46bf-9369-8ef9888e637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:47.775568 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a7f7c-c58e-46bf-9369-8ef9888e637b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:47.850156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:47.891106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:47.891634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:47.891916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:47.892188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:47.893850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:47.984630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:47.986281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c2d23e-5197-416b-9149-e4aec37a78e9 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:48.004204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 710/1416] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:48.894153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d564-1854-4f92-a4ad-cda34100f10f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.894697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d564-1854-4f92-a4ad-cda34100f10f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.894822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7d564-1854-4f92-a4ad-cda34100f10f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.896126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 707/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:48.901527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47f592-aa02-4f7b-9981-ed1fda51f234 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:48.901916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47f592-aa02-4f7b-9981-ed1fda51f234 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:48.902068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a47f592-aa02-4f7b-9981-ed1fda51f234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:48.977909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8a47f592-aa02-4f7b-9981-ed1fda51f234 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:48.979149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8a47f592-aa02-4f7b-9981-ed1fda51f234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:49.005811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:49.076398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 711/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:49.227206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7d564-1854-4f92-a4ad-cda34100f10f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:49.228652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-92d7d564-1854-4f92-a4ad-cda34100f10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:49.376383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7d564-1854-4f92-a4ad-cda34100f10f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:49.419707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:49.496063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:49.496583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:49.496860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:49.497131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:49.498795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:49.590636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:49.592265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5db23-1133-412d-8b9b-338949022770 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:49.611063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 708/1419] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:50.512288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7f71-358e-4c4c-b452-33800ec3317c None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:50.512844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7f71-358e-4c4c-b452-33800ec3317c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:50.512844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7f71-358e-4c4c-b452-33800ec3317c None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:50.514271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 712/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:50.519944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30127a9e-fc5b-41b7-a73b-154468be4b1e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:50.520386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30127a9e-fc5b-41b7-a73b-154468be4b1e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:50.520509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30127a9e-fc5b-41b7-a73b-154468be4b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:50.575245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:50.599869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-30127a9e-fc5b-41b7-a73b-154468be4b1e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:50.601086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-30127a9e-fc5b-41b7-a73b-154468be4b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:50.698404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 709/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:50.850520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d7f71-358e-4c4c-b452-33800ec3317c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:50.851982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-935d7f71-358e-4c4c-b452-33800ec3317c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:51.006054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d7f71-358e-4c4c-b452-33800ec3317c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:51.018026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:51.120417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:51.121024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:51.121311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:51.121579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:51.123224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:51.212192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:51.213795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df89134-5c25-4892-8c12-143dd85fd4f7 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:51.232531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 713/1422] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:52.091424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d789fb-578b-4775-a02c-ecc712207afa None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.091986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d789fb-578b-4775-a02c-ecc712207afa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.091986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d789fb-578b-4775-a02c-ecc712207afa None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.093320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 710/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:52.098440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95112d7e-5afe-4bc3-a2b3-5c9bd4b7b165 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.098877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95112d7e-5afe-4bc3-a2b3-5c9bd4b7b165 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.098973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95112d7e-5afe-4bc3-a2b3-5c9bd4b7b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.121644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.179657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-95112d7e-5afe-4bc3-a2b3-5c9bd4b7b165 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:52.181476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-95112d7e-5afe-4bc3-a2b3-5c9bd4b7b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:52.275277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 714/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:52.425329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d789fb-578b-4775-a02c-ecc712207afa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:52.426900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42d789fb-578b-4775-a02c-ecc712207afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:52.542095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.577535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d789fb-578b-4775-a02c-ecc712207afa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:52.690136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:52.690789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:52.691061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:52.691324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:52.692916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:52.767452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:52.781708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:52.783468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375ef01-9657-448e-8819-8e9b8cb289f0 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:52.801995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 711/1425] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:53.661391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e212c88-f9da-42bf-b213-51afacc43ab6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.661899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e212c88-f9da-42bf-b213-51afacc43ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.661995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e212c88-f9da-42bf-b213-51afacc43ab6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.663325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 715/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:53.669185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e00d3-04b6-4bad-89f7-8cec9778e2bf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:53.669606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e00d3-04b6-4bad-89f7-8cec9778e2bf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:53.669731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e00d3-04b6-4bad-89f7-8cec9778e2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:53.746047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-137e00d3-04b6-4bad-89f7-8cec9778e2bf None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:53.747298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-137e00d3-04b6-4bad-89f7-8cec9778e2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:53.843152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 712/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:53.994614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e212c88-f9da-42bf-b213-51afacc43ab6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:53.996149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2e212c88-f9da-42bf-b213-51afacc43ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:54.079967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:54.144372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e212c88-f9da-42bf-b213-51afacc43ab6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:54.256614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:54.257225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:54.257509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:54.257783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:54.260031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:54.309147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:54.352045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:54.353676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dd1c7-340f-44b3-ba95-1e2b432c37b8 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:54.372133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 716/1428] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:55.180947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3450b11-55e1-47b3-81ea-33c52daf6171 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.181526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3450b11-55e1-47b3-81ea-33c52daf6171 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.181526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3450b11-55e1-47b3-81ea-33c52daf6171 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.182862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 713/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:55.187862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae84d9-0a4d-41e2-aa8e-b1a1e40e5f15 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.188329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae84d9-0a4d-41e2-aa8e-b1a1e40e5f15 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.188440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fae84d9-0a4d-41e2-aa8e-b1a1e40e5f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.266294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7fae84d9-0a4d-41e2-aa8e-b1a1e40e5f15 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:55.267451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7fae84d9-0a4d-41e2-aa8e-b1a1e40e5f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:55.367810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 717/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:55.519834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3450b11-55e1-47b3-81ea-33c52daf6171 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:55.521278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f3450b11-55e1-47b3-81ea-33c52daf6171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:55.571656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:55.669288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3450b11-55e1-47b3-81ea-33c52daf6171 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:55.784302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:55.784818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:55.785092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:55.785361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:55.787005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:55.794245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:55.876506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:55.878111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b79081-ccb1-48aa-a3ed-64ede4e51e60 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:55.895666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 714/1431] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:56.711576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b4a04-a00c-48d7-b616-e07a2a51b8d9 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.712168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b4a04-a00c-48d7-b616-e07a2a51b8d9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.712168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b4a04-a00c-48d7-b616-e07a2a51b8d9 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.713392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 718/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:56.719259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40a3d-efbc-4ccd-ae76-41a82e51e9fb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:56.719259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40a3d-efbc-4ccd-ae76-41a82e51e9fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:56.720180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e40a3d-efbc-4ccd-ae76-41a82e51e9fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:56.801386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-44e40a3d-efbc-4ccd-ae76-41a82e51e9fb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:56.802721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44e40a3d-efbc-4ccd-ae76-41a82e51e9fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:56.873811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:56.904753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 715/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:57.061267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b4a04-a00c-48d7-b616-e07a2a51b8d9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:57.062887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee6b4a04-a00c-48d7-b616-e07a2a51b8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:57.081590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.210142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b4a04-a00c-48d7-b616-e07a2a51b8d9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:57.301662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:57.328112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:57.328710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:57.328985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:57.329252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:57.330880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:57.418783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:57.420371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd136e38-cb6a-424c-a509-519eadd6447e service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:57.438506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 719/1434] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:29:58.244606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e70ab-7d78-4c45-a475-9d558309dc34 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.245214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e70ab-7d78-4c45-a475-9d558309dc34 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.245214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303e70ab-7d78-4c45-a475-9d558309dc34 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.246514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 716/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:29:58.251514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba027-70ad-4188-bbf1-bfcb69335ffd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.251968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba027-70ad-4188-bbf1-bfcb69335ffd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.252081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baba027-70ad-4188-bbf1-bfcb69335ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.329355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2baba027-70ad-4188-bbf1-bfcb69335ffd None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:58.330605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2baba027-70ad-4188-bbf1-bfcb69335ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:58.372731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:58.427666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 720/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:29:58.578446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e70ab-7d78-4c45-a475-9d558309dc34 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:29:58.580014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-303e70ab-7d78-4c45-a475-9d558309dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:58.729823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303e70ab-7d78-4c45-a475-9d558309dc34 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:29:58.783805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:58.845260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:29:58.845868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:58.846141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:58.846445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:58.848038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:58.938448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:29:58.940027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79cae2-df78-4d3d-a578-499f4188c2d0 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:29:58.958417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 717/1437] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:29:59.761230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bf667-cd97-4564-9fe6-45efd9a27435 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.761805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bf667-cd97-4564-9fe6-45efd9a27435 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.761805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bf667-cd97-4564-9fe6-45efd9a27435 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.763157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 721/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:29:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:29:59.768196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8526a-210a-40d9-a4ac-ffe028b39385 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:29:59.768627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8526a-210a-40d9-a4ac-ffe028b39385 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:29:59.768740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8526a-210a-40d9-a4ac-ffe028b39385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:29:59.845468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dfd8526a-210a-40d9-a4ac-ffe028b39385 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:29:59.846695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dfd8526a-210a-40d9-a4ac-ffe028b39385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:29:59.848736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:29:59.945010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 718/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:29:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:00.100270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bf667-cd97-4564-9fe6-45efd9a27435 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:00.101710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-058bf667-cd97-4564-9fe6-45efd9a27435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:00.251221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bf667-cd97-4564-9fe6-45efd9a27435 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:00.277921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:00.363715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:00.364232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:00.364511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:00.364781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:00.366416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:00.462702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:00.464331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d17d2e-6ead-47dd-ab3f-2e82e899854e service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:00.483314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 722/1440] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:02.323186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb414b-bcb7-4130-b659-8a4a07ae0a52 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.323753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb414b-bcb7-4130-b659-8a4a07ae0a52 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.323753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fb414b-bcb7-4130-b659-8a4a07ae0a52 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.325104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 719/1441] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:02.330413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1495-7c8d-4046-a6f8-46a9f9142056 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.330869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1495-7c8d-4046-a6f8-46a9f9142056 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.330968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1c1495-7c8d-4046-a6f8-46a9f9142056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.373390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:02.398114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3c1c1495-7c8d-4046-a6f8-46a9f9142056 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:02.399840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3c1c1495-7c8d-4046-a6f8-46a9f9142056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:02.497614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 723/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:02.789105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb414b-bcb7-4130-b659-8a4a07ae0a52 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:02.790758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-78fb414b-bcb7-4130-b659-8a4a07ae0a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:02.896070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fb414b-bcb7-4130-b659-8a4a07ae0a52 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:02.928646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:02.970763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:02.971372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:02.971655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:02.971925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:02.973602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:03.083514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:03.084867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488efdce-f6e0-4477-8836-79dd72875cb0 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:03.103477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 720/1443] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:05.177060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b72f04-7282-4a13-867c-51f2ccdde912 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.177610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b72f04-7282-4a13-867c-51f2ccdde912 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.177708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b72f04-7282-4a13-867c-51f2ccdde912 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.179043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 724/1444] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:05.184948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3baaa7-5d7d-4a9b-8279-674045e2a228 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.185385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3baaa7-5d7d-4a9b-8279-674045e2a228 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.185494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3baaa7-5d7d-4a9b-8279-674045e2a228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.251273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6e3baaa7-5d7d-4a9b-8279-674045e2a228 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:05.252445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6e3baaa7-5d7d-4a9b-8279-674045e2a228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:05.265408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.354781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 721/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:05.641051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b72f04-7282-4a13-867c-51f2ccdde912 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:05.642740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-11b72f04-7282-4a13-867c-51f2ccdde912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:05.757129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b72f04-7282-4a13-867c-51f2ccdde912 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:05.770543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:05.839308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:05.839972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:05.840261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:05.840536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:05.842236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:05.956797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:05.958281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdad630d-de99-4947-bc9e-9e04fa73ef41 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:05.976823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 725/1446] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:08.218621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778728-f1f6-4389-b54f-019088ad6d0e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.219231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778728-f1f6-4389-b54f-019088ad6d0e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.219231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd778728-f1f6-4389-b54f-019088ad6d0e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.220521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 722/1447] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:08.225815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c645-532c-48e2-95f3-317e8d269727 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.226234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c645-532c-48e2-95f3-317e8d269727 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.226404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1c645-532c-48e2-95f3-317e8d269727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.275397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.293261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-54d1c645-532c-48e2-95f3-317e8d269727 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:08.294510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54d1c645-532c-48e2-95f3-317e8d269727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:08.391652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 726/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:08.674667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd778728-f1f6-4389-b54f-019088ad6d0e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:08.676219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bd778728-f1f6-4389-b54f-019088ad6d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:08.780118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd778728-f1f6-4389-b54f-019088ad6d0e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:08.821541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:08.855037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:08.855649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:08.855930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:08.856204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:08.857833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:08.965603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:08.966982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613f864-0254-4ed8-b749-3eb3c3766d7d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:08.985160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 723/1449] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:11.020698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8cdc7-68fe-4106-ace2-ad2dd715a22f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.021224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8cdc7-68fe-4106-ace2-ad2dd715a22f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.021224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b8cdc7-68fe-4106-ace2-ad2dd715a22f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.022618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 727/1450] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:11.028023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d87a8-c5d9-4c87-96fd-b688870f557d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.028410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d87a8-c5d9-4c87-96fd-b688870f557d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.028538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d87a8-c5d9-4c87-96fd-b688870f557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.095827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cf7d87a8-c5d9-4c87-96fd-b688870f557d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:11.097048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cf7d87a8-c5d9-4c87-96fd-b688870f557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:11.125039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:11.209255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 724/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:11.489386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8cdc7-68fe-4106-ace2-ad2dd715a22f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:11.490973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-20b8cdc7-68fe-4106-ace2-ad2dd715a22f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:11.595270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b8cdc7-68fe-4106-ace2-ad2dd715a22f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:11.615482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:11.670033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:11.670662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:11.670936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:11.671204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:11.672787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:11.784845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:11.786229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9614b375-7f58-497b-9851-fbf624576678 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:11.804465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 728/1452] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:13.796523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6d4a4-d6f0-4df0-943e-37bba30ed873 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.797347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6d4a4-d6f0-4df0-943e-37bba30ed873 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.797347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c6d4a4-d6f0-4df0-943e-37bba30ed873 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.798464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 725/1453] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:13.804437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8d963-2512-4df6-a00f-1ec4a289775c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:13.805194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8d963-2512-4df6-a00f-1ec4a289775c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:13.805194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8d963-2512-4df6-a00f-1ec4a289775c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:13.913019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:13.918581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fcb8d963-2512-4df6-a00f-1ec4a289775c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:13.920714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fcb8d963-2512-4df6-a00f-1ec4a289775c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.043741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 729/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:14.336910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6d4a4-d6f0-4df0-943e-37bba30ed873 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:14.338482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e6c6d4a4-d6f0-4df0-943e-37bba30ed873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.441391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c6d4a4-d6f0-4df0-943e-37bba30ed873 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:14.489603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:14.515750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:14.516352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:14.516633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:14.516904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:14.518843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:14.632057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:14.633512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c5d239-12b4-48a1-8176-c4c1fa068a96 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:14.653613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 726/1455] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:16.642543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982adde-7941-4ff7-bb22-19b958bb1922 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.643091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982adde-7941-4ff7-bb22-19b958bb1922 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.643091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982adde-7941-4ff7-bb22-19b958bb1922 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.644428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 730/1456] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:16.651161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f622f96-379a-4f7e-990e-f66f7b44deb6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:16.653141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f622f96-379a-4f7e-990e-f66f7b44deb6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:16.653141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f622f96-379a-4f7e-990e-f66f7b44deb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:16.734392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3f622f96-379a-4f7e-990e-f66f7b44deb6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:16.735724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3f622f96-379a-4f7e-990e-f66f7b44deb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:16.775875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:16.869014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 727/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:17.168510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982adde-7941-4ff7-bb22-19b958bb1922 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:17.170044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9982adde-7941-4ff7-bb22-19b958bb1922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:17.285309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982adde-7941-4ff7-bb22-19b958bb1922 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:17.314386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:17.376203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:17.376752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:17.377032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:17.377299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:17.378944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:17.511087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:17.512705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab870e3-e873-462b-9250-f259eca117e6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:17.532106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 731/1458] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:19.656028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b47488-fbc2-496f-a0a3-7570cf4a7d2e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.656861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b47488-fbc2-496f-a0a3-7570cf4a7d2e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.656861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b47488-fbc2-496f-a0a3-7570cf4a7d2e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.658024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 728/1459] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:19.664586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f4328-14da-40fa-a128-d37c18da8f48 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:19.665216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f4328-14da-40fa-a128-d37c18da8f48 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:19.665216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f4328-14da-40fa-a128-d37c18da8f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:19.746065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ab2f4328-14da-40fa-a128-d37c18da8f48 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:19.747318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ab2f4328-14da-40fa-a128-d37c18da8f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:19.756203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:19.852393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 732/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:20.141934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b47488-fbc2-496f-a0a3-7570cf4a7d2e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:20.143620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a7b47488-fbc2-496f-a0a3-7570cf4a7d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:20.267000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b47488-fbc2-496f-a0a3-7570cf4a7d2e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:20.326927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:20.346629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:20.347244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:20.347629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:20.347932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:20.349640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:20.478892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:20.480342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36c4714-0d90-4c0c-a43e-0b14370ebacc service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:20.498724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 729/1461] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:22.517840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57fb8d-7e83-4cc5-bf4e-248fc0c91617 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.518387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57fb8d-7e83-4cc5-bf4e-248fc0c91617 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.518485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57fb8d-7e83-4cc5-bf4e-248fc0c91617 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.519809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 733/1462] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:22.527060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155fe48-ab01-4fad-b4e7-50b554f87e7a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:22.527558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155fe48-ab01-4fad-b4e7-50b554f87e7a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:22.528050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a155fe48-ab01-4fad-b4e7-50b554f87e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:22.604619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a155fe48-ab01-4fad-b4e7-50b554f87e7a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:22.605852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a155fe48-ab01-4fad-b4e7-50b554f87e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:22.652793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:22.714872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 730/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:30:22.997466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57fb8d-7e83-4cc5-bf4e-248fc0c91617 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:22.999085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8c57fb8d-7e83-4cc5-bf4e-248fc0c91617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.109224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57fb8d-7e83-4cc5-bf4e-248fc0c91617 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:23.163436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:23.210526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:23.211145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:23.211421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:23.211686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:23.213480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:23.344279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:23.345651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3711004-61ce-4386-b4ba-5ea98fe11697 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:23.365812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 734/1464] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:25.470824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecec58-f41d-4181-bed8-d9a9a27668af None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.471896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecec58-f41d-4181-bed8-d9a9a27668af None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.471896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ecec58-f41d-4181-bed8-d9a9a27668af None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.473409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 731/1465] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:25.479698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e8eb-7281-4f99-bcdb-f5bf7f36ffea None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:25.480268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e8eb-7281-4f99-bcdb-f5bf7f36ffea None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:25.480268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d622e8eb-7281-4f99-bcdb-f5bf7f36ffea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:25.553763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d622e8eb-7281-4f99-bcdb-f5bf7f36ffea None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:25.555012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d622e8eb-7281-4f99-bcdb-f5bf7f36ffea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:25.572979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:25.663456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 735/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:25.968500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecec58-f41d-4181-bed8-d9a9a27668af None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:25.970686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c4ecec58-f41d-4181-bed8-d9a9a27668af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:26.103649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ecec58-f41d-4181-bed8-d9a9a27668af None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:26.193995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:26.203384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:26.204227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:26.204618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:26.205014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:26.207643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:26.340915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:26.342380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42bcc49-b1f0-4faf-b9e8-a277a146d5f8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:26.360459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 732/1467] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:28.438093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033e852-c363-4d10-8b27-9feba7767696 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.438829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033e852-c363-4d10-8b27-9feba7767696 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.439138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033e852-c363-4d10-8b27-9feba7767696 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.440718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 736/1468] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:30:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:28.448071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc60fd-21c0-4549-af56-41a88fbdd53f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:28.448646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc60fd-21c0-4549-af56-41a88fbdd53f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:28.448646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fc60fd-21c0-4549-af56-41a88fbdd53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:28.560565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-65fc60fd-21c0-4549-af56-41a88fbdd53f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:28.562586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65fc60fd-21c0-4549-af56-41a88fbdd53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:28.648980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:28.831754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 733/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:30:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:29.115663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033e852-c363-4d10-8b27-9feba7767696 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:29.117220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c033e852-c363-4d10-8b27-9feba7767696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.230607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033e852-c363-4d10-8b27-9feba7767696 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:29.289364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:29.323203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:29.323877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:29.324232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:29.324597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:29.325924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:29.466079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:29.467579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28408eb-5773-4753-9f4e-e8cf396b44a6 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:29.494578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 737/1470] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:30:38.829794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=22 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:39.488690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=66 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:50.569753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371b0a-7369-444e-b240-088d49de0ae8 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.570355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371b0a-7369-444e-b240-088d49de0ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.570700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4371b0a-7369-444e-b240-088d49de0ae8 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.573327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 734/1471] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:30:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:50.579143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911693-320b-4f5a-b18e-60d689bc364b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:50.579885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911693-320b-4f5a-b18e-60d689bc364b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:50.580112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78911693-320b-4f5a-b18e-60d689bc364b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:50.683847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-78911693-320b-4f5a-b18e-60d689bc364b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:50.685252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-78911693-320b-4f5a-b18e-60d689bc364b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:50.791131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 738/1472] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:51.096202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da8567-4695-4eb0-afb1-ed28367fe0c2 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.096652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da8567-4695-4eb0-afb1-ed28367fe0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.096821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da8567-4695-4eb0-afb1-ed28367fe0c2 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.098252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 735/1473] 10.222.0.91 () {54 vars in 967 bytes} [Wed Aug 24 03:30:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:51.105021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169cef-90cf-4882-9389-e0282c646b1a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.105553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169cef-90cf-4882-9389-e0282c646b1a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.105920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b169cef-90cf-4882-9389-e0282c646b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.201477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9b169cef-90cf-4882-9389-e0282c646b1a None None] MFA Rules not processed for user `7d0eef76ae53400c83c391351dc75c32`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:51.202778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9b169cef-90cf-4882-9389-e0282c646b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.226931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.295833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 739/1474] 10.222.0.91 () {58 vars in 1064 bytes} [Wed Aug 24 03:30:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:51.303842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da8567-4695-4eb0-afb1-ed28367fe0c2 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:51.305384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-93da8567-4695-4eb0-afb1-ed28367fe0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.420043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da8567-4695-4eb0-afb1-ed28367fe0c2 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:51.516395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:51.517355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:51.517785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:51.518245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:51.521026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:51.613406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:51.679922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:51.681292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda2cc34-5b9a-4a57-95f0-35c2dd2a6569 service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:51.701165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 736/1475] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:30:53.836795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d9871-ebb5-440e-a510-e135402c97de None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.837385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d9871-ebb5-440e-a510-e135402c97de None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.837534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d9871-ebb5-440e-a510-e135402c97de None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.839387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 740/1476] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:30:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:30:53.846167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1d745-94d7-4dc8-8650-c6f53a05bb0e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:53.846995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1d745-94d7-4dc8-8650-c6f53a05bb0e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:53.847160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1d745-94d7-4dc8-8650-c6f53a05bb0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:53.917002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ddf1d745-94d7-4dc8-8650-c6f53a05bb0e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:53.918355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ddf1d745-94d7-4dc8-8650-c6f53a05bb0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.083229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 737/1477] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:54.395909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d9871-ebb5-440e-a510-e135402c97de None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:54.397505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-425d9871-ebb5-440e-a510-e135402c97de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:54.552534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d9871-ebb5-440e-a510-e135402c97de None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:54.749986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:54.833177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:54.834107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:54.834604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:54.835044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:54.836081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:54.839035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:30:54.839035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:30:54.839035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:30:54.840995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c4f825-723c-4f2c-ae4c-a8e8fb54b62d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:54.873132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 741/1478] 10.222.0.91 () {56 vars in 1108 bytes} [Wed Aug 24 03:30:54 2022] GET /identity/v3/projects => generated 2553 bytes in 478 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:30:56.889729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b37a3f-2399-451e-a1c5-b28ce4908def None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.890370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b37a3f-2399-451e-a1c5-b28ce4908def None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.890370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b37a3f-2399-451e-a1c5-b28ce4908def None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.891557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 738/1479] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:30:56.897452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c06f-69de-4f43-afc8-16baef6189b1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:56.898118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c06f-69de-4f43-afc8-16baef6189b1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:56.898118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f0c06f-69de-4f43-afc8-16baef6189b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:56.980424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-46f0c06f-69de-4f43-afc8-16baef6189b1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:30:56.981662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-46f0c06f-69de-4f43-afc8-16baef6189b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.122043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 742/1480] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:30:57.434226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b37a3f-2399-451e-a1c5-b28ce4908def None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:30:57.435886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d3b37a3f-2399-451e-a1c5-b28ce4908def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.446179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.598119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b37a3f-2399-451e-a1c5-b28ce4908def None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:30:57.698754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:30:57.736470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:30:57.737284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:30:57.737553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:30:57.737817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:30:57.739583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:30:57.845060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:30:57.846694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f772e4-16c3-4b9a-9b51-ce78200d548b service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:30:57.865456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 739/1481] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:00.105232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ca79a-0737-4d1b-ba8b-7fe9a39aa530 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.106051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ca79a-0737-4d1b-ba8b-7fe9a39aa530 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.106340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ca79a-0737-4d1b-ba8b-7fe9a39aa530 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.108304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 743/1482] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:31:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:00.114944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fad56-bd52-42f0-ba9c-48e76b2a8382 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.115547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fad56-bd52-42f0-ba9c-48e76b2a8382 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.115547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14fad56-bd52-42f0-ba9c-48e76b2a8382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.216255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d14fad56-bd52-42f0-ba9c-48e76b2a8382 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:00.217619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d14fad56-bd52-42f0-ba9c-48e76b2a8382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.279528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.351040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 740/1483] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:00.666984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ca79a-0737-4d1b-ba8b-7fe9a39aa530 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:00.669058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-064ca79a-0737-4d1b-ba8b-7fe9a39aa530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:00.716446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:00.848492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ca79a-0737-4d1b-ba8b-7fe9a39aa530 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:00.938360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:00.939102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:00.939494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:00.939811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:00.941774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:01.035842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:01.076483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:01.077974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c115aed0-2094-4fee-94a7-8551e82f96dd service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:01.100841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 744/1484] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:03.243448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaec31-e487-4938-8f81-c10382a65aee None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.244651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaec31-e487-4938-8f81-c10382a65aee None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.244980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aaec31-e487-4938-8f81-c10382a65aee None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.247660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 741/1485] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:31:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:03.254038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bfaba-0824-49eb-b0c7-736dc5310447 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:03.254657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bfaba-0824-49eb-b0c7-736dc5310447 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:03.254824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5bfaba-0824-49eb-b0c7-736dc5310447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:03.345704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dd5bfaba-0824-49eb-b0c7-736dc5310447 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:03.347073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dd5bfaba-0824-49eb-b0c7-736dc5310447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:03.460094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 745/1486] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:03.763705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaec31-e487-4938-8f81-c10382a65aee None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:03.765248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-12aaec31-e487-4938-8f81-c10382a65aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:03.902810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aaec31-e487-4938-8f81-c10382a65aee None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:03.998134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:04.001283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:04.001283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:04.001847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:04.002424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:04.005709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:04.143947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:04.145628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79584bda-6e55-4067-9bc3-70391a363492 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:04.164753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 742/1487] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:06.232105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0bf08-4123-4fa1-8aa6-9af7127e3c32 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.233014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0bf08-4123-4fa1-8aa6-9af7127e3c32 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.233014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b0bf08-4123-4fa1-8aa6-9af7127e3c32 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.233986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 746/1488] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:31:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:06.240071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e630d0-bec5-4c93-8ff2-45d46f588ae4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:06.240801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e630d0-bec5-4c93-8ff2-45d46f588ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:06.240801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e630d0-bec5-4c93-8ff2-45d46f588ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:06.324820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-08e630d0-bec5-4c93-8ff2-45d46f588ae4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:06.326067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-08e630d0-bec5-4c93-8ff2-45d46f588ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.393454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.452963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 743/1489] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:31:06.765799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0bf08-4123-4fa1-8aa6-9af7127e3c32 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:06.767420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-16b0bf08-4123-4fa1-8aa6-9af7127e3c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:06.819558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:06.915810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b0bf08-4123-4fa1-8aa6-9af7127e3c32 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:07.010811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:07.011338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:07.011602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:07.011865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:07.013484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:07.101556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:07.138285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:07.139747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af7452-d971-49e2-9981-9549eb5b6ae0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:07.158703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 747/1490] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:31:09.410805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194ea19-cf5f-4b5c-ac88-52283a27617d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.411430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194ea19-cf5f-4b5c-ac88-52283a27617d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.411430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3194ea19-cf5f-4b5c-ac88-52283a27617d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.412721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 744/1491] 10.222.0.91 () {54 vars in 927 bytes} [Wed Aug 24 03:31:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:31:09.419942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d0cc0-11ec-4045-953f-12f42444fa6c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:09.420592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d0cc0-11ec-4045-953f-12f42444fa6c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:09.420723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429d0cc0-11ec-4045-953f-12f42444fa6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:09.500319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-429d0cc0-11ec-4045-953f-12f42444fa6c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:31:09.501601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-429d0cc0-11ec-4045-953f-12f42444fa6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:09.614005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 748/1492] 10.222.0.91 () {58 vars in 1024 bytes} [Wed Aug 24 03:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:31:09.936844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194ea19-cf5f-4b5c-ac88-52283a27617d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:31:09.938464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3194ea19-cf5f-4b5c-ac88-52283a27617d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:10.054826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3194ea19-cf5f-4b5c-ac88-52283a27617d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:31:10.153584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:31:10.154228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:31:10.154491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:31:10.154754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:31:10.156562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:31:10.167668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:10.296083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:31:10.298472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1091259a-4e68-449a-9d9b-4cd3685e5e97 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:31:10.327430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 745/1493] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:31:19.611795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:31:20.323307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=50 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:32.331796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:32.332490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:32.332490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:32.423083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:32.424559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:32.532578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 749/1494] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:32:32.541791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:32.543241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:32.543241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:32.637274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:32.638516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:32.751618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 746/1495] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:32:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:32:32.757992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:32.759720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:32.777274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f893d-420e-4f5d-84cb-cf0b8dc9e247 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:32.864032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:32.911528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:32.912142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:32.912420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:32.912684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:32.913338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:32:32.913745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:32.914924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:32.914924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:32:32.914924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:32:32.915826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:32.928724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 750/1496] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:32:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:32:32.942687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:32.952523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218290a-6f27-4074-99fd-a9c89541a5ce None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.020552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:33.058245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.058997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.059379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.059747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.060476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.062518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:33.062518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:32:33.062518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:32:33.064167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.132172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 747/1497] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:32:32 2022] POST /identity/v3/projects => generated 377 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:32:33.136952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:33.138824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:33.173625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151179d5-f562-407c-8846-6b3eb3bfa508 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.274521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.275037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.275313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.275584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.276229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.277927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:33.277927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:32:33.277927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:32:33.279196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.324532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 751/1498] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:32:33 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:32:33.330576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:33.333139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:33.404348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f8b27c-5a1a-4f86-81fe-b031ebb52610 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.499822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:33.512148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.512758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.513030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.513296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.513948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.515004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:33.515004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:32:33.515004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:32:33.515888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.530556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 748/1499] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:32:33.535294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:33.536825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:33.548502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a787b11f-4f01-40d5-a36d-a5074103f0b4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.576174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:33.670571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.671498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.671943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.672386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.673471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.675447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:33.675447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:32:33.675447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:32:33.676932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d36f9a-8769-4560-b044-4dc498d49c8a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.699700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 752/1500] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:32:33 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:32:33.704673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:33.707380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:33.753624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598b1dd-815c-4bbd-99da-f72ca89d8638 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:33.873600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:33.874177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.874548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.874888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] PATH_INFO: `/v3/projects/0ebbf980311c4c5b94a8c23896001501/users/deec78b769284ab6b92d7e1da2a3ba74/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:33.911525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ebbf980311c4c5b94a8c23896001501, user_id=deec78b769284ab6b92d7e1da2a3ba74, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:33.912770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:33.912770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:32:33.912770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:32:33.915698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbdbb45-ef89-4a12-8001-c840d3bb0846 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:33.956112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 749/1501] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:32:33 2022] PUT /identity/v3/projects/0ebbf980311c4c5b94a8c23896001501/users/deec78b769284ab6b92d7e1da2a3ba74/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:33.969337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6383d5-c864-457e-b239-41cc2f7b847e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:33.969790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6383d5-c864-457e-b239-41cc2f7b847e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:33.969934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6383d5-c864-457e-b239-41cc2f7b847e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:34.049404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ac6383d5-c864-457e-b239-41cc2f7b847e None None] MFA Rules not processed for user `deec78b769284ab6b92d7e1da2a3ba74`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:34.050684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ac6383d5-c864-457e-b239-41cc2f7b847e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:34.136863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 753/1502] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:32:33 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:32:34.150706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b2152-18b2-4619-aa75-d7f05d884414 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:34.151040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b2152-18b2-4619-aa75-d7f05d884414 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:34.151387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3b2152-18b2-4619-aa75-d7f05d884414 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:34.152823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 750/1503] 10.222.0.91 () {54 vars in 964 bytes} [Wed Aug 24 03:32:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:32:34.157865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4bb17-2ba5-4029-8875-402ec920d820 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:34.158150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4bb17-2ba5-4029-8875-402ec920d820 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:34.158584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf4bb17-2ba5-4029-8875-402ec920d820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:34.201609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:34.262689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5bf4bb17-2ba5-4029-8875-402ec920d820 None None] MFA Rules not processed for user `408b15d1a55a439c93a2f879c0d1cc31`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:34.263936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5bf4bb17-2ba5-4029-8875-402ec920d820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:34.344934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 754/1504] 10.222.0.91 () {58 vars in 1061 bytes} [Wed Aug 24 03:32:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:34.353011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b2152-18b2-4619-aa75-d7f05d884414 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:34.354599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1f3b2152-18b2-4619-aa75-d7f05d884414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:34.412008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:34.494444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3b2152-18b2-4619-aa75-d7f05d884414 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:34.585839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:34.586479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:34.586834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:34.587647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:34.588723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:34.699469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:34.700800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:34.704725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:34.720599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 751/1505] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:32:34.777632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f9103-0bcd-45b0-91ee-7206dcf444a8 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:34.778211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f9103-0bcd-45b0-91ee-7206dcf444a8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:34.778211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f9103-0bcd-45b0-91ee-7206dcf444a8 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:34.779492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 755/1506] 10.222.0.91 () {52 vars in 830 bytes} [Wed Aug 24 03:32:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:32:34.783790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:34.785039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:34.904610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3a8e3b-e241-4934-b7d8-8e5a2550814e service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:35.013206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:35.014088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.014513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.015473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] PATH_INFO: `/v3/users/deec78b769284ab6b92d7e1da2a3ba74` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:35.031755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deec78b769284ab6b92d7e1da2a3ba74)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:35.032800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:35.032800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:32:35.032800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:32:35.034573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:35.046523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:35.118641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deec78b769284ab6b92d7e1da2a3ba74'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:35.119210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deec78b769284ab6b92d7e1da2a3ba74'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:35.119660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deec78b769284ab6b92d7e1da2a3ba74'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:35.130949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deec78b769284ab6b92d7e1da2a3ba74'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:35.156268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deec78b769284ab6b92d7e1da2a3ba74'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:35.176430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 752/1507] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:32:34 2022] DELETE /identity/v3/users/deec78b769284ab6b92d7e1da2a3ba74 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:35.181512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.181790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.182082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:35.240843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:32:35.242452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:35.323112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:35.336253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 756/1508] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:32:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:32:35.346870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:35.348207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:35.456860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c439aea3-a531-41bb-9e56-530ef8bbbbf2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:35.517463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:35.541755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:35.542961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:35.543240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:35.543511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:35.545191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:35.663852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:35.665297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0cfa4b-68d8-413a-ab1c-0cae95279021 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:35.688285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 753/1509] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:32:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:32:36.080372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:32:36.080372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:32:36.100928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993bfbcb-586b-4897-9f27-067b6d21d26d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:32:36.202474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:32:36.203009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:32:36.203290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:32:36.203557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] PATH_INFO: `/v3/projects/0ebbf980311c4c5b94a8c23896001501` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:32:36.213366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ebbf980311c4c5b94a8c23896001501)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:32:36.214989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:32:36.214989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:32:36.214989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:32:36.216182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:32:36.277534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ebbf980311c4c5b94a8c23896001501', 'request_id': 'req-89d86a21-a73d-404e-a2d9-8c262b0bf58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:36.283701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ebbf980311c4c5b94a8c23896001501', 'request_id': 'req-89d86a21-a73d-404e-a2d9-8c262b0bf58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:36.293063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ebbf980311c4c5b94a8c23896001501', 'request_id': 'req-89d86a21-a73d-404e-a2d9-8c262b0bf58b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:36.328561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] The token cache is being invalidate because project 0ebbf980311c4c5b94a8c23896001501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:32:36.328870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:36.329430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-89d86a21-a73d-404e-a2d9-8c262b0bf58b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:32:36.330946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 757/1510] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:32:36 2022] DELETE /identity/v3/projects/0ebbf980311c4c5b94a8c23896001501 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:32:45.685936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=62 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:32:46.326583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:12.816701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5aae9-3dc8-4908-93ed-5e214523afc5 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.817352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5aae9-3dc8-4908-93ed-5e214523afc5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.817352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d5aae9-3dc8-4908-93ed-5e214523afc5 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.818679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 754/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:34:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:12.823610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c20d-a0a9-4edc-837e-26a49c1d6e6f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:12.824097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c20d-a0a9-4edc-837e-26a49c1d6e6f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:12.824211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280c20d-a0a9-4edc-837e-26a49c1d6e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:12.909448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b280c20d-a0a9-4edc-837e-26a49c1d6e6f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:12.910721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b280c20d-a0a9-4edc-837e-26a49c1d6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:13.013540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 758/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:13.167281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5aae9-3dc8-4908-93ed-5e214523afc5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:13.169909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-30d5aae9-3dc8-4908-93ed-5e214523afc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:13.290476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d5aae9-3dc8-4908-93ed-5e214523afc5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:13.296691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4058f7-622e-41ee-81e7-f11f3c6ce58b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:13.297300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4058f7-622e-41ee-81e7-f11f3c6ce58b service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:13.297572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4058f7-622e-41ee-81e7-f11f3c6ce58b service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:13.297838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4058f7-622e-41ee-81e7-f11f3c6ce58b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:13.306282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4058f7-622e-41ee-81e7-f11f3c6ce58b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:13.307856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4058f7-622e-41ee-81e7-f11f3c6ce58b service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:13.317554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 755/1513] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:14.159178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228a306-b0d5-48ae-9f49-175304209947 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:14.159726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228a306-b0d5-48ae-9f49-175304209947 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:14.159726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7228a306-b0d5-48ae-9f49-175304209947 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:14.161108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 759/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:34:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:14.165901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a49d58-92aa-4c79-b46b-fea1071f1b09 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:14.166390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a49d58-92aa-4c79-b46b-fea1071f1b09 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:14.166498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a49d58-92aa-4c79-b46b-fea1071f1b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:14.186966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f4a49d58-92aa-4c79-b46b-fea1071f1b09 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:14.188169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4a49d58-92aa-4c79-b46b-fea1071f1b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:14.195876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 756/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:14.347574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228a306-b0d5-48ae-9f49-175304209947 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:14.362829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7228a306-b0d5-48ae-9f49-175304209947 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:14.368189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3bae3a-5b53-4d08-8ca2-c87c221e7aa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:14.368779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bae3a-5b53-4d08-8ca2-c87c221e7aa7 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:14.369056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bae3a-5b53-4d08-8ca2-c87c221e7aa7 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:14.369327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3bae3a-5b53-4d08-8ca2-c87c221e7aa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:14.377420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bae3a-5b53-4d08-8ca2-c87c221e7aa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:14.379101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3bae3a-5b53-4d08-8ca2-c87c221e7aa7 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:14.388314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 760/1516] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:16.542542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb945a1-64bf-45ec-9423-f7ce5727bd31 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.543132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb945a1-64bf-45ec-9423-f7ce5727bd31 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.543132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb945a1-64bf-45ec-9423-f7ce5727bd31 None None] PATH_INFO: `/v3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.545059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 757/1517] 10.222.0.75 () {54 vars in 925 bytes} [Wed Aug 24 03:34:16 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:34:16.553867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada126c-6733-4617-b4f9-582ab733e6f1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:16.554338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada126c-6733-4617-b4f9-582ab733e6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:16.554482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ada126c-6733-4617-b4f9-582ab733e6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:16.618730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9ada126c-6733-4617-b4f9-582ab733e6f1 None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:16.620000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9ada126c-6733-4617-b4f9-582ab733e6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:16.636901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:16.695198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 761/1518] 10.222.0.75 () {58 vars in 1015 bytes} [Wed Aug 24 03:34:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:24.181554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=56 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:25.650710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfa030-b497-4a5b-bf3f-ae88139573cb None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.651398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfa030-b497-4a5b-bf3f-ae88139573cb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.651736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acfa030-b497-4a5b-bf3f-ae88139573cb None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.654008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 758/1519] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:34:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:25.660907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8ba7-ead5-4ce6-9491-e22ad77d8636 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.661331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8ba7-ead5-4ce6-9491-e22ad77d8636 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.661755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f8ba7-ead5-4ce6-9491-e22ad77d8636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.689745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-479f8ba7-ead5-4ce6-9491-e22ad77d8636 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:25.691007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-479f8ba7-ead5-4ce6-9491-e22ad77d8636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:25.699142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 762/1520] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:25.855533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfa030-b497-4a5b-bf3f-ae88139573cb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:25.872718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acfa030-b497-4a5b-bf3f-ae88139573cb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:25.880150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e60aa6-8669-461f-b755-02e6fcdb7c4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:25.881136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e60aa6-8669-461f-b755-02e6fcdb7c4d service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:25.881615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e60aa6-8669-461f-b755-02e6fcdb7c4d service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:25.881913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e60aa6-8669-461f-b755-02e6fcdb7c4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:25.892086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e60aa6-8669-461f-b755-02e6fcdb7c4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:25.893703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e60aa6-8669-461f-b755-02e6fcdb7c4d service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:25.905278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 759/1521] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:34:26.743652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a06b-93bc-494c-abf6-712ad4a8af4f None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.744215 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a06b-93bc-494c-abf6-712ad4a8af4f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.744215 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a9a06b-93bc-494c-abf6-712ad4a8af4f None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.745551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 763/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:34:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:26.750748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a7953-17e6-4e3e-a6c6-979a9cca8e2d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.751515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a7953-17e6-4e3e-a6c6-979a9cca8e2d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.751515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a7953-17e6-4e3e-a6c6-979a9cca8e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.773133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9d3a7953-17e6-4e3e-a6c6-979a9cca8e2d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:26.774530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9d3a7953-17e6-4e3e-a6c6-979a9cca8e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:26.790158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 760/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:34:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:26.954117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a06b-93bc-494c-abf6-712ad4a8af4f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:26.969204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a9a06b-93bc-494c-abf6-712ad4a8af4f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:26.976119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923b179-92a5-41a4-98d1-77e3c02bce6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:26.976734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923b179-92a5-41a4-98d1-77e3c02bce6b service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:26.977007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923b179-92a5-41a4-98d1-77e3c02bce6b service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:26.977276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923b179-92a5-41a4-98d1-77e3c02bce6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:26.986553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923b179-92a5-41a4-98d1-77e3c02bce6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:26.988171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923b179-92a5-41a4-98d1-77e3c02bce6b service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:26.998248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 764/1524] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:28.194883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a2e0a-3573-4d01-ae77-2838655f6a3e None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:28.195419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a2e0a-3573-4d01-ae77-2838655f6a3e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:28.195419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108a2e0a-3573-4d01-ae77-2838655f6a3e None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:28.196758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 761/1525] 10.222.0.91 () {54 vars in 961 bytes} [Wed Aug 24 03:34:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:28.202310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff5d3b-1aef-4cc5-a74f-e9f65f750ee7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:28.202893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff5d3b-1aef-4cc5-a74f-e9f65f750ee7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:28.203042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff5d3b-1aef-4cc5-a74f-e9f65f750ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:28.222080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0fff5d3b-1aef-4cc5-a74f-e9f65f750ee7 None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:28.223330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0fff5d3b-1aef-4cc5-a74f-e9f65f750ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:28.232552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 765/1526] 10.222.0.91 () {58 vars in 1058 bytes} [Wed Aug 24 03:34:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:28.240375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a2e0a-3573-4d01-ae77-2838655f6a3e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:28.254561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108a2e0a-3573-4d01-ae77-2838655f6a3e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:28.259427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5b33e7-4998-4848-b821-cdb3b5772b37 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:28.260015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b33e7-4998-4848-b821-cdb3b5772b37 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:28.260287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b33e7-4998-4848-b821-cdb3b5772b37 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:28.260557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5b33e7-4998-4848-b821-cdb3b5772b37 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:28.268784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b33e7-4998-4848-b821-cdb3b5772b37 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:28.270379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5b33e7-4998-4848-b821-cdb3b5772b37 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:28.280046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 762/1527] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:38.217369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=35 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:34:38.246085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6042a-60b7-4a56-8132-827866410b67 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:38.246464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6042a-60b7-4a56-8132-827866410b67 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:38.246814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a6042a-60b7-4a56-8132-827866410b67 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:38.248230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 766/1528] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:34:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:38.253481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddde37-a7f4-407f-a5a3-7114dcbe15a3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:38.254041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddde37-a7f4-407f-a5a3-7114dcbe15a3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:38.254041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ddde37-a7f4-407f-a5a3-7114dcbe15a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:38.274279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b2ddde37-a7f4-407f-a5a3-7114dcbe15a3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:38.275548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b2ddde37-a7f4-407f-a5a3-7114dcbe15a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:38.283669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 763/1529] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:34:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:38.446029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6042a-60b7-4a56-8132-827866410b67 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:38.463033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a6042a-60b7-4a56-8132-827866410b67 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:38.468690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7416aef-1579-40da-8e38-07bc19cf0bc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:38.469297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416aef-1579-40da-8e38-07bc19cf0bc9 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:38.469573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416aef-1579-40da-8e38-07bc19cf0bc9 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:38.469833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7416aef-1579-40da-8e38-07bc19cf0bc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:38.483530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7416aef-1579-40da-8e38-07bc19cf0bc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:38.487740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7416aef-1579-40da-8e38-07bc19cf0bc9 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:38.499305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 767/1530] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:39.335637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055ff32-6d56-4bcd-8fc8-eddb36bcd59d None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:39.336213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055ff32-6d56-4bcd-8fc8-eddb36bcd59d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:39.336213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055ff32-6d56-4bcd-8fc8-eddb36bcd59d None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:39.337492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 764/1531] 127.0.0.1 () {54 vars in 901 bytes} [Wed Aug 24 03:34:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:39.343066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442921c-8fe9-4681-ab79-975959d05850 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:39.343769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442921c-8fe9-4681-ab79-975959d05850 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:39.343769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5442921c-8fe9-4681-ab79-975959d05850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:39.364654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5442921c-8fe9-4681-ab79-975959d05850 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:39.365778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5442921c-8fe9-4681-ab79-975959d05850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:39.374377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 768/1532] 127.0.0.1 () {58 vars in 998 bytes} [Wed Aug 24 03:34:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:39.527077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055ff32-6d56-4bcd-8fc8-eddb36bcd59d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:39.542923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055ff32-6d56-4bcd-8fc8-eddb36bcd59d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:39.548597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5c244-86f1-414a-a0c2-2afc1e25d1df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:39.549203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5c244-86f1-414a-a0c2-2afc1e25d1df service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:39.549479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5c244-86f1-414a-a0c2-2afc1e25d1df service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:39.549745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5c244-86f1-414a-a0c2-2afc1e25d1df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:39.558542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5c244-86f1-414a-a0c2-2afc1e25d1df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:39.560361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5c244-86f1-414a-a0c2-2afc1e25d1df service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:39.570348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 765/1533] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:34:43.674316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9d960-0014-422d-a31c-327b980f3fd6 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:43.674912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9d960-0014-422d-a31c-327b980f3fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:43.674912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac9d960-0014-422d-a31c-327b980f3fd6 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:43.676253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 769/1534] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:34:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:34:43.681610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd666a-89a5-461d-ad20-1501d24b6588 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:43.682020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd666a-89a5-461d-ad20-1501d24b6588 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:43.682147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecd666a-89a5-461d-ad20-1501d24b6588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:43.703238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0ecd666a-89a5-461d-ad20-1501d24b6588 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:43.704496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ecd666a-89a5-461d-ad20-1501d24b6588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:43.712321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 766/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:34:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:44.001343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9d960-0014-422d-a31c-327b980f3fd6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:44.003075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6ac9d960-0014-422d-a31c-327b980f3fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:44.083013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac9d960-0014-422d-a31c-327b980f3fd6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:44.086599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88dfb0c-57ec-416c-ab7c-5e0845cdcfcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:44.087208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dfb0c-57ec-416c-ab7c-5e0845cdcfcf service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:44.087489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dfb0c-57ec-416c-ab7c-5e0845cdcfcf service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:44.087759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88dfb0c-57ec-416c-ab7c-5e0845cdcfcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:44.101659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dfb0c-57ec-416c-ab7c-5e0845cdcfcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:44.103162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88dfb0c-57ec-416c-ab7c-5e0845cdcfcf service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:44.113238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 770/1536] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:46.036993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e83f6-e107-4761-b648-c7cd638df6b7 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:46.037780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e83f6-e107-4761-b648-c7cd638df6b7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:46.037780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98e83f6-e107-4761-b648-c7cd638df6b7 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:46.038958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 767/1537] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:34:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:46.044987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de30f0c-fb17-41c1-b02b-1867566e2fcb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:46.045448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de30f0c-fb17-41c1-b02b-1867566e2fcb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:46.045596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de30f0c-fb17-41c1-b02b-1867566e2fcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:46.065734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6de30f0c-fb17-41c1-b02b-1867566e2fcb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:46.066996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6de30f0c-fb17-41c1-b02b-1867566e2fcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:46.075579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 771/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:46.359919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e83f6-e107-4761-b648-c7cd638df6b7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:46.370929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98e83f6-e107-4761-b648-c7cd638df6b7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:46.374732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e7ccf4-22bd-40fa-a15b-d3feda9b83a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:46.375337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7ccf4-22bd-40fa-a15b-d3feda9b83a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:46.375617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7ccf4-22bd-40fa-a15b-d3feda9b83a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:46.375889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e7ccf4-22bd-40fa-a15b-d3feda9b83a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:46.385823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7ccf4-22bd-40fa-a15b-d3feda9b83a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:46.387323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e7ccf4-22bd-40fa-a15b-d3feda9b83a5 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:46.397266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 768/1539] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:50.008838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fbe77-e606-4e72-9de9-ff53ce35d320 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:50.009438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fbe77-e606-4e72-9de9-ff53ce35d320 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:50.009438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fbe77-e606-4e72-9de9-ff53ce35d320 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:50.011305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 772/1540] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:34:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:50.016854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702feb-eff2-4492-90a8-53424a1c2157 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:50.017387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702feb-eff2-4492-90a8-53424a1c2157 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:50.017558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9702feb-eff2-4492-90a8-53424a1c2157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:50.038389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d9702feb-eff2-4492-90a8-53424a1c2157 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:50.039601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d9702feb-eff2-4492-90a8-53424a1c2157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:50.047662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 769/1541] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:50.607584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fbe77-e606-4e72-9de9-ff53ce35d320 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:50.609463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-121fbe77-e606-4e72-9de9-ff53ce35d320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:50.704424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fbe77-e606-4e72-9de9-ff53ce35d320 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:50.708379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07832b-d2ce-4556-98d0-f761909c5047 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:50.709001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07832b-d2ce-4556-98d0-f761909c5047 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:50.709280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07832b-d2ce-4556-98d0-f761909c5047 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:50.709550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07832b-d2ce-4556-98d0-f761909c5047 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:50.719557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07832b-d2ce-4556-98d0-f761909c5047 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:50.720983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07832b-d2ce-4556-98d0-f761909c5047 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:50.732015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 773/1542] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:34:52.720448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003b514-f9a2-4f4c-bd2f-7dbadc7c709b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:52.721024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003b514-f9a2-4f4c-bd2f-7dbadc7c709b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:52.721024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003b514-f9a2-4f4c-bd2f-7dbadc7c709b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:52.722363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 770/1543] 127.0.0.1 () {54 vars in 917 bytes} [Wed Aug 24 03:34:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:34:52.728030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a4e22-6efd-4c90-9b6c-28e378246f19 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:52.728600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a4e22-6efd-4c90-9b6c-28e378246f19 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:52.729229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a4e22-6efd-4c90-9b6c-28e378246f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:52.748609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-054a4e22-6efd-4c90-9b6c-28e378246f19 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:34:52.749907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-054a4e22-6efd-4c90-9b6c-28e378246f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:52.758828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 774/1544] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Aug 24 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:34:53.325970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003b514-f9a2-4f4c-bd2f-7dbadc7c709b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:34:53.327702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4003b514-f9a2-4f4c-bd2f-7dbadc7c709b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:34:53.340569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003b514-f9a2-4f4c-bd2f-7dbadc7c709b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:34:53.344977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9ec95-9e5c-4ce3-91d8-419364eb8e51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:34:53.345754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9ec95-9e5c-4ce3-91d8-419364eb8e51 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:34:53.346443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9ec95-9e5c-4ce3-91d8-419364eb8e51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:34:53.346443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9ec95-9e5c-4ce3-91d8-419364eb8e51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:34:53.356546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9ec95-9e5c-4ce3-91d8-419364eb8e51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:34:53.357960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9ec95-9e5c-4ce3-91d8-419364eb8e51 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:34:53.367432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 771/1545] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:34:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:35:02.743701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:35:03.363508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=42 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:23.085175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95d7f2-b5b6-4057-97a6-0d029b751b03 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.086219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95d7f2-b5b6-4057-97a6-0d029b751b03 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.086219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95d7f2-b5b6-4057-97a6-0d029b751b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.115776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-be95d7f2-b5b6-4057-97a6-0d029b751b03 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.117021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-be95d7f2-b5b6-4057-97a6-0d029b751b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.131391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 775/1546] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:23.136783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80126f8a-5bd6-4074-9bf5-045c120677c4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.137369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80126f8a-5bd6-4074-9bf5-045c120677c4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.137369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80126f8a-5bd6-4074-9bf5-045c120677c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.154057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6cc7-e74f-4e61-bfd3-e6d134c084f6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.154057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6cc7-e74f-4e61-bfd3-e6d134c084f6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.154057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc6cc7-e74f-4e61-bfd3-e6d134c084f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.165787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-80126f8a-5bd6-4074-9bf5-045c120677c4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.167406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-80126f8a-5bd6-4074-9bf5-045c120677c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.180639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a1cc6cc7-e74f-4e61-bfd3-e6d134c084f6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.184524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 772/1547] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.186213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1129-a334-4b87-9439-a7842c1191fe None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.187197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1129-a334-4b87-9439-a7842c1191fe None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.187684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ea1129-a334-4b87-9439-a7842c1191fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.188020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a1cc6cc7-e74f-4e61-bfd3-e6d134c084f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.209750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 776/1548] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.222118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0c77-bfc1-4184-bb16-f5021f1b2616 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.223121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0c77-bfc1-4184-bb16-f5021f1b2616 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.223953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073c0c77-bfc1-4184-bb16-f5021f1b2616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.258346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-54ea1129-a334-4b87-9439-a7842c1191fe None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.258346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-54ea1129-a334-4b87-9439-a7842c1191fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.271342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 773/1549] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.273623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209f34-d613-43b1-9e4f-725aa4645291 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.273899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209f34-d613-43b1-9e4f-725aa4645291 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.274171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e209f34-d613-43b1-9e4f-725aa4645291 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.291341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-073c0c77-bfc1-4184-bb16-f5021f1b2616 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.292616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-073c0c77-bfc1-4184-bb16-f5021f1b2616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.315703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8e209f34-d613-43b1-9e4f-725aa4645291 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.316973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8e209f34-d613-43b1-9e4f-725aa4645291 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.329571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 774/1550] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.330922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.331207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.331476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.343953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 777/1551] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.345486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c4c49-21b8-40fa-beee-f067e53c249c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.345763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c4c49-21b8-40fa-beee-f067e53c249c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.346033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793c4c49-21b8-40fa-beee-f067e53c249c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.359563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.361150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.377427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-793c4c49-21b8-40fa-beee-f067e53c249c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.378694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-793c4c49-21b8-40fa-beee-f067e53c249c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.383886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 775/1552] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.385307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.395886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 778/1553] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.398148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.398457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.398729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.407550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafa6d0c-3d51-43cb-8262-65eec07d0bd1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.414757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.415404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.415684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.415954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.416628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:23.416936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.419069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:36:23.419069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:36:23.419069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:36:23.420346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.424964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.426589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.436045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 776/1554] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:23.436781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 779/1555] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.437014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.437855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.449678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed543a-bd98-45bf-8a7a-2c91b6b7cf6e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.458423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.459404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.459690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.459961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.460617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:23.460918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.461467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c7a979-8acb-49d7-a1d4-0d4ef63bcedb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.462690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.466684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.468085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.468375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.468642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.469441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:23.469749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.471385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78ca8b-e31c-4d7e-a322-632eb5e0fbb9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.475990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 777/1556] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:23.479552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.488557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 780/1557] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:23.489771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1976-4cfd-4e7f-a81f-11901a080896 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.490061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1976-4cfd-4e7f-a81f-11901a080896 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.494207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb1976-4cfd-4e7f-a81f-11901a080896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.494369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175a347c-4437-4838-93f1-380c70ae9a1a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.499752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.500380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.500655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.500931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.501580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:23.501884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.503533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.513720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-63eb1976-4cfd-4e7f-a81f-11901a080896 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.514901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-63eb1976-4cfd-4e7f-a81f-11901a080896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.524978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 778/1558] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:23.524978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.527633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 781/1559] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.529249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718006bc-f0e1-4f29-80f3-f91cf8aa2cfc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.529526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718006bc-f0e1-4f29-80f3-f91cf8aa2cfc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.529800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718006bc-f0e1-4f29-80f3-f91cf8aa2cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.540276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b8892-a18e-446d-8c32-46a0228a8cc3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.545405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.546114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.546919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.547200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.547828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.549730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.553788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-718006bc-f0e1-4f29-80f3-f91cf8aa2cfc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.555025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-718006bc-f0e1-4f29-80f3-f91cf8aa2cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.565882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 782/1560] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.570819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475d24-368c-494e-8444-0832262fe82d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.571097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475d24-368c-494e-8444-0832262fe82d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.571367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed475d24-368c-494e-8444-0832262fe82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.598034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ed475d24-368c-494e-8444-0832262fe82d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.599378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ed475d24-368c-494e-8444-0832262fe82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.609332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 783/1561] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.610356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed475d24-368c-494e-8444-0832262fe82d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.617848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 779/1562] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:23.618716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.624326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed475d24-368c-494e-8444-0832262fe82d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.692748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.693252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a50e4f-bbad-4a4d-82c2-9345f70db413 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.693663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.694078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.694384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.695041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.696198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:36:23.696198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:36:23.696198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:36:23.697412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.699385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dcc38a-a07d-4dd1-bc14-6c93266793b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.699939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc38a-a07d-4dd1-bc14-6c93266793b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.700376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc38a-a07d-4dd1-bc14-6c93266793b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.700661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dcc38a-a07d-4dd1-bc14-6c93266793b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.701301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc38a-a07d-4dd1-bc14-6c93266793b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.703338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dcc38a-a07d-4dd1-bc14-6c93266793b0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.729258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 784/1563] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/projects => generated 379 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:23.730089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.734677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 780/1564] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:23.735914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.736209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.736483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.757656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.758915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.805142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09fb20-c62a-47cb-a739-a6944b1332ee admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.805895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:23.827511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 781/1565] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.828909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.830706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:23.840796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798eefa5-1d37-486f-8a45-57cd1768d4af None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.845566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.846174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.846484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.846753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.847397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:23.847696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.849287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bda8b9-5da9-4e2e-be24-c915890ec17c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.861862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 782/1566] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:23.863169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.863440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.863716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.871135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5526ac5d-e947-4c53-8ec9-9bae2aef8a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.871895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ac5d-e947-4c53-8ec9-9bae2aef8a80 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.872273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ac5d-e947-4c53-8ec9-9bae2aef8a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.872645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5526ac5d-e947-4c53-8ec9-9bae2aef8a80 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.873385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526ac5d-e947-4c53-8ec9-9bae2aef8a80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.875467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5526ac5d-e947-4c53-8ec9-9bae2aef8a80 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.884883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.886128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.894234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 783/1567] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:23.895094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.906231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ae8b3-abb1-4e9a-b888-1d5742b14af8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.912904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 785/1568] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/projects => generated 387 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:23.913107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:23.913229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.913229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.913229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.913455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:23.913455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:23.913455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:23.913455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:23.914388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:36:23.914388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:36:23.914388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:36:23.915368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:23.933470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:23.934842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:23.944321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 784/1569] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/users => generated 293 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:23.945263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:23.956071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae6a3-1ec5-457a-98d1-536e524b5699 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:23.999177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 786/1570] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:24.003069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.014534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4e05c5-9454-4a1b-b106-95dbff39a216 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.017249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.017868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.018147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.018451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.019055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.019219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.019653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.019930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.020196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.020821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.020975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.022810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.042527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 785/1571] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.043369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.048684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 787/1572] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:23 2022] POST /identity/v3/users => generated 282 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.048684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.054104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfd4aab-cd57-4b77-8603-d3f35972de29 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.058805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.059349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.059622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.059888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.060549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:24.060851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.062458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3ea29-cba4-4c4d-a0ba-02a07e2fa9ce admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.064983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b41b04a-b332-4f45-82e5-634d6a22eb13 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.070328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.070938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.071217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.071492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.072157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.074099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.077457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 786/1573] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:24.079117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.079409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.079988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.100849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:24.103100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:24.108828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 788/1574] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.108828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.172726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 787/1575] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:24.173654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.175672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6558b-ba5f-41aa-9884-8752a02660e8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.181837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.182518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.182828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.183131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.183776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.184162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26925510-b38f-4f28-9dcd-9ce57e08771b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.185760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.188760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.189371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.189664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.189914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.190588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.192311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.207228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 788/1576] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.208104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.210416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 789/1577] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.211270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.218693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35f9504-cb42-42e8-9d6e-3f6fd8a6c798 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.220042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:24.227015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.227343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bada151d-e31c-4958-9082-bf5b56a6b691 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.227645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.227928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.228201 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.229021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:24.229327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.231275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.232869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.233566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.233821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.234089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.234771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.236372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.244994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 789/1578] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:24.245672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.250659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 790/1579] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.250659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.252736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:24.257614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa80065-1692-4a9c-8849-2cd26287ec05 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.263389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.263590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae7bbbd-3735-4b2d-aa49-04e2e99986e1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.263999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.264276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.264552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.265192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.266834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.268657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.269261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.269539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.269891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.270558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.273210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.278537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 790/1580] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.280032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.290590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d417e021-37f4-4344-aa66-0b6cdd2fea8b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.295376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.295978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.296254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.296516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.297395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.299332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.307946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 791/1581] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/projects => generated 393 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.307946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.325492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 791/1582] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.325492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.340135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ae9ce4-2f24-4fed-ac79-1412744594de admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.372338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e812a02-abed-4478-a8e9-49093306b5a9 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.377261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.377869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.378152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.378460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.379116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:24.379423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.381580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.393418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 792/1583] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:24.394364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.405400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b076be-d77c-467f-822b-1a9494b40cc0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.408270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.408897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.409181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.409469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.410132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.411660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.411804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.412351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.412633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.412907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.413559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.415205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.422647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 792/1584] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.423519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.427747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 793/1585] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.428577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.435779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa6cfaa-a374-4add-8a45-3aeb7c19c048 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.438817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa061a6-fc14-48b2-bff6-0b4154f03701 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.440545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.441156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.441440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.441712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.442386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.443988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.444498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.444620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.444844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.445101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.445845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.447653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.455612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 793/1586] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.456653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.467539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d3e989-e296-426d-b7bf-1b8b3c35f4ea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.472467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.473194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.473583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.473936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.474765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.476501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.478353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 794/1587] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.479625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.487247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 794/1588] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.487977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.498168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf6710-d84d-431b-84d0-62564e86f1ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.546385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bcd2e0-c57e-4ad3-a701-7728a23a826a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.551378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.551993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.552276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.552552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.553317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.555214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.556753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.557474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.557867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.558267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.559013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.560697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.574126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 795/1589] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.578045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.578504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 795/1590] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.579320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.588800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2267b3-be06-49cc-acab-e4d6f4fdea80 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.591629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f4568-eda8-438c-a7a0-d666c9d73420 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.593421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.594031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.594341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.594612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.595275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.596672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.597174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.597333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.597626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.597904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] PATH_INFO: `/v3/projects/c7c94d613f6b440bae60fc442661c671/users/b1d9d5fd0db54ddf85e05c77dd7f6c89/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.613090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c94d613f6b440bae60fc442661c671, user_id=b1d9d5fd0db54ddf85e05c77dd7f6c89, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.614141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:36:24.614141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:36:24.614141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:36:24.616298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.619607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:24.626502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 796/1591] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.627162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.635264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 796/1592] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:24 2022] PUT /identity/v3/projects/c7c94d613f6b440bae60fc442661c671/users/b1d9d5fd0db54ddf85e05c77dd7f6c89/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:24.636082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.655654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:24.696007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1705747d-0a0c-43fc-8094-def0be123d8f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.700912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.701667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.701948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.702242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.702891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.704495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.704737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492304b-5f2c-4bb5-af9d-445ebf996ec6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.710647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.711310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.711587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.712027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] PATH_INFO: `/v3/projects/53aa6850ae4d44619cc27cef9dcadc61/users/7f1047b63c4148ca8073e3414ef84d28/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.714972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 797/1593] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.715864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.727651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53aa6850ae4d44619cc27cef9dcadc61, user_id=7f1047b63c4148ca8073e3414ef84d28, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.728255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7691fc95-bfd3-4fa5-827e-dc0af8bfd4ac admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.730438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.733166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.734220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.734507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.734825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.735434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.737467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.744708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 797/1594] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:24 2022] PUT /identity/v3/projects/53aa6850ae4d44619cc27cef9dcadc61/users/7f1047b63c4148ca8073e3414ef84d28/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:24.745654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.767848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 798/1595] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/users => generated 288 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.767848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.776110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13340b97-430a-4f36-a80e-41eff74362ef admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.815409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c3778-a640-4c14-af7e-8d98cb42f7f3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.820876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.821548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.821832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.822113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] PATH_INFO: `/v3/projects/c2d6d16a54894909ae5571b4dcaf18fa/users/ecde8dea673d40b2abf4c0a0237da3cb/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.834603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.835303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.835584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.835855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.836528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.836931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2d6d16a54894909ae5571b4dcaf18fa, user_id=ecde8dea673d40b2abf4c0a0237da3cb, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.838124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.839653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.848943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 799/1596] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.853299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.855248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 798/1597] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:24 2022] PUT /identity/v3/projects/c2d6d16a54894909ae5571b4dcaf18fa/users/ecde8dea673d40b2abf4c0a0237da3cb/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:24.855940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.863093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4aac1f-dfde-429f-b516-b277f384300b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.928731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3f2634-ad81-461d-9099-1329279bde59 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.929004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0df33fa-bf90-4341-bc8d-85fa9cc0cf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.929526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df33fa-bf90-4341-bc8d-85fa9cc0cf1c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.929789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df33fa-bf90-4341-bc8d-85fa9cc0cf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.930062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0df33fa-bf90-4341-bc8d-85fa9cc0cf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.930750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df33fa-bf90-4341-bc8d-85fa9cc0cf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.932378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0df33fa-bf90-4341-bc8d-85fa9cc0cf1c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.933339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.933941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.934234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.934512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.935135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:24.937035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:24.942671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 800/1598] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:24.944111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa9f52-c165-4a2b-b39c-a9a57eba9343 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.944394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa9f52-c165-4a2b-b39c-a9a57eba9343 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.944665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fa9f52-c165-4a2b-b39c-a9a57eba9343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.946762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:24.964942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 799/1599] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/users => generated 302 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:24.964942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:24.965273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-69fa9f52-c165-4a2b-b39c-a9a57eba9343 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:24.965939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-69fa9f52-c165-4a2b-b39c-a9a57eba9343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:24.973449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739209f-c3bd-4f5b-99f1-4d72e0febcba admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:24.974031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 801/1600] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:24.975936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.976217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.976492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:24.986322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:24.987183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:24.987600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:24.987943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] PATH_INFO: `/v3/projects/6f0007a8402e4889bc8ed99a507036e4/users/0b294c0072b740cc960bc53e1f806ffc/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.000152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:25.001450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:25.010719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f0007a8402e4889bc8ed99a507036e4, user_id=0b294c0072b740cc960bc53e1f806ffc, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.012993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 802/1601] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:25.013452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.014170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.029858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 800/1602] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:24 2022] PUT /identity/v3/projects/6f0007a8402e4889bc8ed99a507036e4/users/0b294c0072b740cc960bc53e1f806ffc/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:25.030862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.033290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa879215-5b1b-4d04-b016-206854d9e6b6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.056719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:25.103130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680df39b-ceca-490f-81db-9752a7a5c6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.104021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df39b-ceca-490f-81db-9752a7a5c6ed admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.104295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df39b-ceca-490f-81db-9752a7a5c6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.104564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680df39b-ceca-490f-81db-9752a7a5c6ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.105213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680df39b-ceca-490f-81db-9752a7a5c6ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.106891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680df39b-ceca-490f-81db-9752a7a5c6ed admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.108710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450b996-a5d6-491a-833f-ec59d4a4a996 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.113413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.114072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.114384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.114662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] PATH_INFO: `/v3/projects/1ce5ae57e2664ca1a071f3a148c0cfb1/users/d7a8af6c549841a7a2f690fb2f642b7a/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.117930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 803/1603] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:25.119209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.119493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.119765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.133843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ce5ae57e2664ca1a071f3a148c0cfb1, user_id=d7a8af6c549841a7a2f690fb2f642b7a, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.137977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.151361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] MFA Rules not processed for user `ecde8dea673d40b2abf4c0a0237da3cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:25.152737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:25.155116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 801/1604] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:25 2022] PUT /identity/v3/projects/1ce5ae57e2664ca1a071f3a148c0cfb1/users/d7a8af6c549841a7a2f690fb2f642b7a/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:25.163338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.220642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 804/1605] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:25.221656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.237837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b83b2-a2ad-4154-81dd-0aead7adaec8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.243613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.244909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.245275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.245626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.246367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.247989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.259528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 802/1606] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:25.260474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.298645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fd075-47e7-4f74-a2fb-4346cf664db1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.303604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.304251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.304716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.304902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.305434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.307124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.315682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:25.317360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 805/1607] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:25.318222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.322030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce18ec6-c30d-40b1-af04-b1a7b68d2eb2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.326919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddcaea-f305-428a-9378-21023f01ae57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.327460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddcaea-f305-428a-9378-21023f01ae57 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.327892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddcaea-f305-428a-9378-21023f01ae57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.328202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddcaea-f305-428a-9378-21023f01ae57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.329597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750310d9-446d-4c21-84e5-0e2f9c56251d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.334855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.335589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.335948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.336276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.339677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddcaea-f305-428a-9378-21023f01ae57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.341419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ddcaea-f305-428a-9378-21023f01ae57 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.346754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.348288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.353220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 803/1608] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:25.354441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7b25-d7f0-49c8-8884-59278fadb06f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.354727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7b25-d7f0-49c8-8884-59278fadb06f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.355002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97e7b25-d7f0-49c8-8884-59278fadb06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.359361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 806/1609] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:25.360388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.370049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcc2984-b7dc-49ec-a021-3f97f49e260e service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.375463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.376148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.376502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.376795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a97e7b25-d7f0-49c8-8884-59278fadb06f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:25.376916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.377615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.378259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a97e7b25-d7f0-49c8-8884-59278fadb06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:25.379390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.387293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 804/1610] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:25.388446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.388941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.389176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.395358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 807/1611] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:25.396455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.409005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c197c1b-5473-4727-9c1b-d7972d385b7c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.416861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.417538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.417974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.418405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.419141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.421287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.426482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] MFA Rules not processed for user `d7a8af6c549841a7a2f690fb2f642b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:25.428651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:25.447180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:25.460014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 808/1612] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/projects => generated 365 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:25.461099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.527102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 805/1613] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:25.528970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.540367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf0954-daf3-41f8-8cb4-1c0adcdf0df1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.545993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.547098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.547498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.547827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960cda9-7963-4011-88a7-b7de7e47b7e5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.547945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] PATH_INFO: `/v3/projects/9494dfc650374d64bf4b5f46259a0171/users/76c0df49fa9342778a56eae098b2f2cd/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.556281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.557915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.558889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.559623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.561585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.565153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.569238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9494dfc650374d64bf4b5f46259a0171, user_id=76c0df49fa9342778a56eae098b2f2cd, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.571910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.579612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 806/1614] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:25.580835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.590810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 809/1615] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:25 2022] PUT /identity/v3/projects/9494dfc650374d64bf4b5f46259a0171/users/76c0df49fa9342778a56eae098b2f2cd/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:25.591849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.593367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c77f11d-b78c-462f-b98d-04f2b327eee8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.596052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:25.669073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c8e4ee-b2fe-4285-906e-255e79235bf2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.671125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.672143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.672489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.672837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] PATH_INFO: `/v3/projects/d1e90a027f97405bb3b955ed2c51b109/users/5a87acbd9ba24431a280b9f67913dd57/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.675189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.676106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.676388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.676776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.689695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.692165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.694259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1e90a027f97405bb3b955ed2c51b109, user_id=5a87acbd9ba24431a280b9f67913dd57, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.697424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.707519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 810/1616] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:25.715492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 807/1617] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:25 2022] PUT /identity/v3/projects/d1e90a027f97405bb3b955ed2c51b109/users/5a87acbd9ba24431a280b9f67913dd57/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:25.716642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.718528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.730371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fbab09-5eb9-43df-ae32-cdb445b3b4e1 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.790274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:25.809632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ed76b6-5be1-440b-9b34-dc85a02c7978 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.810178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.811221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.811573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.811901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.812594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.813715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5e925-76d0-4236-81bc-455db072ecf6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.814501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5e925-76d0-4236-81bc-455db072ecf6 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.814936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.815054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5e925-76d0-4236-81bc-455db072ecf6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.815549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5e925-76d0-4236-81bc-455db072ecf6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.853565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 811/1618] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/users => generated 289 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:25.853565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:25.866992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8fed6d-c966-4b95-9f31-7cc17fc56f39 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:25.875354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c907bf-fc68-4b9d-a050-725d047888fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:25.881129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c907bf-fc68-4b9d-a050-725d047888fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.881615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c907bf-fc68-4b9d-a050-725d047888fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.882791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c907bf-fc68-4b9d-a050-725d047888fc admin admin] PATH_INFO: `/v3/projects/f9dba1cb24744d9fac9203b61e06df36/users/2799f24651554e328107623e134e76c9/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.915183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5e925-76d0-4236-81bc-455db072ecf6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.919904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb5e925-76d0-4236-81bc-455db072ecf6 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.933323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:25.948522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c907bf-fc68-4b9d-a050-725d047888fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9dba1cb24744d9fac9203b61e06df36, user_id=2799f24651554e328107623e134e76c9, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:25.954052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c907bf-fc68-4b9d-a050-725d047888fc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:25.956639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 808/1619] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:25 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:25.959865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.961587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.961955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:25.979669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 812/1620] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:25 2022] PUT /identity/v3/projects/f9dba1cb24744d9fac9203b61e06df36/users/2799f24651554e328107623e134e76c9/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:25.981985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50d8d1-bb61-4871-b64c-71f5cfad8d37 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:25.982514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50d8d1-bb61-4871-b64c-71f5cfad8d37 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:25.983136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e50d8d1-bb61-4871-b64c-71f5cfad8d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.010347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:26.012084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.061436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6e50d8d1-bb61-4871-b64c-71f5cfad8d37 None None] MFA Rules not processed for user `5a87acbd9ba24431a280b9f67913dd57`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:26.062805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6e50d8d1-bb61-4871-b64c-71f5cfad8d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.204168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 809/1621] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:26.222347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.242110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 813/1622] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:26.243507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bb590-a2a6-4ffa-abbc-594ec811d3d0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.243795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bb590-a2a6-4ffa-abbc-594ec811d3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.244065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77bb590-a2a6-4ffa-abbc-594ec811d3d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.265590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d330ba-e6ae-4c2c-b939-9e84f1a9e5a2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.281968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.282634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.282909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.283175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.284639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.292038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.303687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e77bb590-a2a6-4ffa-abbc-594ec811d3d0 None None] MFA Rules not processed for user `2799f24651554e328107623e134e76c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:26.305662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e77bb590-a2a6-4ffa-abbc-594ec811d3d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.323686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 810/1623] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:26.323686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.467872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403e082e-ed0d-4f99-a02a-f88cbd25f6ef admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.473181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 814/1624] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:26.481032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693575-06d7-44c5-8aff-923501d46a84 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.481302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693575-06d7-44c5-8aff-923501d46a84 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.481577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66693575-06d7-44c5-8aff-923501d46a84 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.484582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.485207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.485469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.485733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.486630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 815/1625] 10.222.0.91 () {54 vars in 962 bytes} [Wed Aug 24 03:36:26 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:26.487456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66693575-06d7-44c5-8aff-923501d46a84 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.514749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.516415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.520986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66693575-06d7-44c5-8aff-923501d46a84 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.532041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518e1cc-74c3-4dc3-9a67-85c9f7d7331a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.533637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518e1cc-74c3-4dc3-9a67-85c9f7d7331a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.534990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518e1cc-74c3-4dc3-9a67-85c9f7d7331a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.534990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518e1cc-74c3-4dc3-9a67-85c9f7d7331a admin admin] PATH_INFO: `/v3/projects/61f5708c8ba34217914d0b8f7c7ac712/users/4bd3e16f20c9436fb5f2da5ffd4b4233/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.543277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 811/1626] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:26.546794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.567487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e4fdc4-4303-470c-82cc-c5b1f3c3aa26 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.571148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518e1cc-74c3-4dc3-9a67-85c9f7d7331a admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f5708c8ba34217914d0b8f7c7ac712, user_id=4bd3e16f20c9436fb5f2da5ffd4b4233, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.574218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5518e1cc-74c3-4dc3-9a67-85c9f7d7331a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.582800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.583516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.583842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.584192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.585637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.589213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.601627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 816/1627] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:26 2022] PUT /identity/v3/projects/61f5708c8ba34217914d0b8f7c7ac712/users/4bd3e16f20c9436fb5f2da5ffd4b4233/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:26.604381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.604667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.604938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.649549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 812/1628] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:26 2022] POST /identity/v3/projects => generated 405 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:26.656753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.679647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:26.697145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:26.700472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:26.771194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:26.831263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f6c0cc-1cc2-427f-9e37-b9dc87a86fa0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.844477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.846478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.846876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.847259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.849046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.852420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.887583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 817/1629] 10.222.0.91 () {58 vars in 1059 bytes} [Wed Aug 24 03:36:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:26.888703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.893042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 813/1630] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:26 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:26.895214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:26.933426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff985e2-9e77-4d01-ad47-d036c3c8b7a1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.936505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f7dba9-f874-426f-82fe-f4c60b002c01 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:26.946566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011a7e46-7435-4992-bf1d-aed7592f4921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.947237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a7e46-7435-4992-bf1d-aed7592f4921 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.947647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a7e46-7435-4992-bf1d-aed7592f4921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.947957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011a7e46-7435-4992-bf1d-aed7592f4921 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.949080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a7e46-7435-4992-bf1d-aed7592f4921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.953123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:26.953821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:26.954873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:26.955148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] PATH_INFO: `/v3/projects/61f5708c8ba34217914d0b8f7c7ac712/users/4bd3e16f20c9436fb5f2da5ffd4b4233/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:26.957456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011a7e46-7435-4992-bf1d-aed7592f4921 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.960328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61f5708c8ba34217914d0b8f7c7ac712, user_id=4bd3e16f20c9436fb5f2da5ffd4b4233, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:26.970133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:26.997577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 814/1631] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:26 2022] PUT /identity/v3/projects/61f5708c8ba34217914d0b8f7c7ac712/users/4bd3e16f20c9436fb5f2da5ffd4b4233/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:26.998581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.022253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 818/1632] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:26 2022] POST /identity/v3/users => generated 309 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:27.022253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.022253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.022253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.151152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] MFA Rules not processed for user `4bd3e16f20c9436fb5f2da5ffd4b4233`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:27.152347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:27.254534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671114fe-ffed-48fa-8598-24f201a69a23 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.262872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.264181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.264675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.265104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.373181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 819/1633] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:27.374060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.396603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.399846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.420107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 815/1634] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:27.421011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.460749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf159cb-1cd7-41f3-a14c-3234bea39f08 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.466311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.467112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.467618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.467618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.468397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.470040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.481348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:27.488166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 820/1635] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:27.490813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.504640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301353d-a5e1-47f5-bbd1-3ffe0c9967f1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.511307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.511986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.512291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.512579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] PATH_INFO: `/v3/projects/e0bd6eef0ea2465d8e65de4bd002c3a3/users/275b46ab9eb143d497512244b6110b6d/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.529853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c2821-79bc-454f-b60b-ac0fe56e33f5 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.535761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bd6eef0ea2465d8e65de4bd002c3a3, user_id=275b46ab9eb143d497512244b6110b6d, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.536826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.538417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.539157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.539157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.539890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.562110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.564336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.573381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 821/1636] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:27 2022] PUT /identity/v3/projects/e0bd6eef0ea2465d8e65de4bd002c3a3/users/275b46ab9eb143d497512244b6110b6d/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:27.574584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.578258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 816/1637] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:27 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:27.592997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:27.730534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df004d3b-f7c4-4d93-838c-2b97b312955d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.736105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85f5ff8-0c55-4185-b1df-e3c19f8ed03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.737169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f5ff8-0c55-4185-b1df-e3c19f8ed03e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.737628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f5ff8-0c55-4185-b1df-e3c19f8ed03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.738154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85f5ff8-0c55-4185-b1df-e3c19f8ed03e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.739032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f5ff8-0c55-4185-b1df-e3c19f8ed03e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.741519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85f5ff8-0c55-4185-b1df-e3c19f8ed03e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.766710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 822/1638] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:27 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:27.767569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:27.787621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95e7c6-f388-4dfd-afbb-47f2275bdb61 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:27.816047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:27.817435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.817973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.818641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] PATH_INFO: `/v3/projects/e0bd6eef0ea2465d8e65de4bd002c3a3/users/275b46ab9eb143d497512244b6110b6d/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.822010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bd6eef0ea2465d8e65de4bd002c3a3, user_id=275b46ab9eb143d497512244b6110b6d, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:27.825987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:27.855713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 817/1639] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:27 2022] PUT /identity/v3/projects/e0bd6eef0ea2465d8e65de4bd002c3a3/users/275b46ab9eb143d497512244b6110b6d/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:27.858073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d297-ab34-4603-882b-5c4a5ccc6fb8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:27.858642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d297-ab34-4603-882b-5c4a5ccc6fb8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:27.859076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724d297-ab34-4603-882b-5c4a5ccc6fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:27.916810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1724d297-ab34-4603-882b-5c4a5ccc6fb8 None None] MFA Rules not processed for user `275b46ab9eb143d497512244b6110b6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:27.918509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1724d297-ab34-4603-882b-5c4a5ccc6fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.064101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 823/1640] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:27 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:28.072587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.222213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd446738-78e0-463c-992e-1adfbca7b7fb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.233885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d383beae-9b53-4e78-9067-92a6457a015e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.234700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383beae-9b53-4e78-9067-92a6457a015e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.236886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383beae-9b53-4e78-9067-92a6457a015e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.239631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d383beae-9b53-4e78-9067-92a6457a015e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.263786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383beae-9b53-4e78-9067-92a6457a015e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.266440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d383beae-9b53-4e78-9067-92a6457a015e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.298716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 818/1641] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:28 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:28.516534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fd232-6209-4340-9947-8896fc29b17f None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.517208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fd232-6209-4340-9947-8896fc29b17f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.517208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7fd232-6209-4340-9947-8896fc29b17f None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.518995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 824/1642] 10.222.0.91 () {54 vars in 962 bytes} [Wed Aug 24 03:36:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:28.543020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.543307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.543891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.602326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:28.603590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:28.688807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fd232-6209-4340-9947-8896fc29b17f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.783249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 819/1643] 10.222.0.91 () {58 vars in 1059 bytes} [Wed Aug 24 03:36:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:28.785304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:28.858670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7fd232-6209-4340-9947-8896fc29b17f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.870051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.872189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.872472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.873276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] PATH_INFO: `/v3/users/2799f24651554e328107623e134e76c9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.876317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2799f24651554e328107623e134e76c9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.882026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:28.931971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.962165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:28.965569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce758f7a-6655-4689-9d94-586c869fdeee None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:28.978521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:28.980662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:28.983934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:28.985504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] PATH_INFO: `/v3/users/ecde8dea673d40b2abf4c0a0237da3cb` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:28.988932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecde8dea673d40b2abf4c0a0237da3cb)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:28.991357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: WARNING py.warnings [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:36:28.991357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: warnings.warn(msg) Aug 24 03:36:28.991357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: Aug 24 03:36:28.994476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.008965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2799f24651554e328107623e134e76c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.010122 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2799f24651554e328107623e134e76c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.012864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2799f24651554e328107623e134e76c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.038783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2799f24651554e328107623e134e76c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.071610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5c448baa-be63-418e-9346-a57f7644d1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2799f24651554e328107623e134e76c9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.108136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 825/1644] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:28 2022] DELETE /identity/v3/users/2799f24651554e328107623e134e76c9 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:29.111786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088003-bbcb-4c12-bb6a-f66a3e861dca None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.111786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088003-bbcb-4c12-bb6a-f66a3e861dca None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.111786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98088003-bbcb-4c12-bb6a-f66a3e861dca None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.112561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 826/1645] 10.222.0.91 () {54 vars in 912 bytes} [Wed Aug 24 03:36:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:29.113753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c987-3c62-4d90-b7b2-2ec4719df7d8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.114030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c987-3c62-4d90-b7b2-2ec4719df7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.114331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8445c987-3c62-4d90-b7b2-2ec4719df7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.147853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8445c987-3c62-4d90-b7b2-2ec4719df7d8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:29.149921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8445c987-3c62-4d90-b7b2-2ec4719df7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.187694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecde8dea673d40b2abf4c0a0237da3cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.189542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecde8dea673d40b2abf4c0a0237da3cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.191163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecde8dea673d40b2abf4c0a0237da3cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.221431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecde8dea673d40b2abf4c0a0237da3cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.239959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f9ba0951-4d4f-4fe5-b744-33609cec8bd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecde8dea673d40b2abf4c0a0237da3cb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:29.311759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 827/1646] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:29.314595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 820/1647] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:28 2022] DELETE /identity/v3/users/ecde8dea673d40b2abf4c0a0237da3cb => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:29.316440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5d618-1f72-4a08-80b8-0dd0a2405439 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.316727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5d618-1f72-4a08-80b8-0dd0a2405439 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.317001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5d618-1f72-4a08-80b8-0dd0a2405439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.319893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.320351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.320696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.355256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:29.356725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.360995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-89a5d618-1f72-4a08-80b8-0dd0a2405439 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:29.361950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-89a5d618-1f72-4a08-80b8-0dd0a2405439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.551366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 821/1648] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:29.559785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:29.566767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.605435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 828/1649] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:29.608342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3aa97-fad8-4349-9ec9-8a923dd612cf None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.612517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3aa97-fad8-4349-9ec9-8a923dd612cf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.613390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d3aa97-fad8-4349-9ec9-8a923dd612cf None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.620859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 829/1650] 10.222.0.91 () {54 vars in 912 bytes} [Wed Aug 24 03:36:29 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:29.622293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.625461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.627083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.678175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:29.680013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.698856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69afccc-bcc6-4baa-9feb-40024a1e1cb6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:29.710000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 830/1651] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:29 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:29.710299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:29.711727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:29.712023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.713244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.713521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.714056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.731743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:29.736274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.754827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:29.762172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 822/1652] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:29.763233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:29.765124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.784037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f22330-003f-40e9-8cbb-050d2f14f342 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:29.793617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:29.794375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.794778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.795050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.808730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad16ca3-259c-49a0-a221-d678eef6cc61 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:29.815403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:29.816327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.816855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.817195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.818654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:29.820823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.831929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:29.833708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.853406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 831/1653] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:29.854844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 823/1654] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:29.855083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:29.857094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:29.886713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855f2f0-5d13-495c-a9b5-89bc7f38bf9e service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:29.889926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:29.890344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:29.891816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:29.892126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:29.911536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:29.914682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:29.937565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 832/1655] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:30.208299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:30.210659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:30.220673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:30.232584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3a2a90-a31b-4115-a8d0-aeea94014a82 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:30.240179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:30.241311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:30.241894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:30.242716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] PATH_INFO: `/v3/users/5a87acbd9ba24431a280b9f67913dd57` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:30.244422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a87acbd9ba24431a280b9f67913dd57)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:30.247501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:30.354993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a87acbd9ba24431a280b9f67913dd57'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.356924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a87acbd9ba24431a280b9f67913dd57'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.376120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a87acbd9ba24431a280b9f67913dd57'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.389918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:30.392690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:30.402535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a87acbd9ba24431a280b9f67913dd57'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.421481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0844d85a-f317-4869-aeae-bd77b21d0da2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a87acbd9ba24431a280b9f67913dd57'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.472045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bb45c8-ed1c-44c2-8802-d5f1b9681152 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:30.480768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:30.486689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 824/1656] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:30 2022] DELETE /identity/v3/users/5a87acbd9ba24431a280b9f67913dd57 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:30.487999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:30.488379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:30.488756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:30.489002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:30.490816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:30.491521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] PATH_INFO: `/v3/projects/f9dba1cb24744d9fac9203b61e06df36` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:30.502953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9dba1cb24744d9fac9203b61e06df36)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:30.508578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: WARNING py.warnings [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 24 03:36:30.508578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: warnings.warn(msg) Aug 24 03:36:30.508578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: Aug 24 03:36:30.514569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:30.535061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:30.548237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:30.690854 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9dba1cb24744d9fac9203b61e06df36', 'request_id': 'req-72ae08f7-a409-49fd-b852-fc75e7097969'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.710926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9dba1cb24744d9fac9203b61e06df36', 'request_id': 'req-72ae08f7-a409-49fd-b852-fc75e7097969'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.747010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 825/1657] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:30.753897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9dba1cb24744d9fac9203b61e06df36', 'request_id': 'req-72ae08f7-a409-49fd-b852-fc75e7097969'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.757560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:30.759758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:30.788479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] The token cache is being invalidate because project f9dba1cb24744d9fac9203b61e06df36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:30.789855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.791972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:30.797943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 833/1658] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:30 2022] DELETE /identity/v3/projects/f9dba1cb24744d9fac9203b61e06df36 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:30.806735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:30.811457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:30.916103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e63ae-7291-47f8-a221-29e35c1697e7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:31.007870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae08f7-a409-49fd-b852-fc75e7097969 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:31.025314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:31.026075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:31.028544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:31.029300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:31.033451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] PATH_INFO: `/v3/users/275b46ab9eb143d497512244b6110b6d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:31.038800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] RBAC: Authorizing `identity:delete_user(user_id=275b46ab9eb143d497512244b6110b6d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:31.041750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:31.089899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:31.104753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:31.104753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:31.104753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:31.111552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:31.136932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:31.138677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:31.157244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 826/1659] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:31.219915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:31.232024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '275b46ab9eb143d497512244b6110b6d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:31.233271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '275b46ab9eb143d497512244b6110b6d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:31.240662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '275b46ab9eb143d497512244b6110b6d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:31.253552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '275b46ab9eb143d497512244b6110b6d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:31.286098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '275b46ab9eb143d497512244b6110b6d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:31.329249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 834/1660] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:30 2022] DELETE /identity/v3/users/275b46ab9eb143d497512244b6110b6d => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:32.663966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.665670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.719106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186df164-f4be-4cfa-91b9-afb42d1ac6e2 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:32.784810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:32.786515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:32.835968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:32.836638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:32.836919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:32.837187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] PATH_INFO: `/v3/projects/c2d6d16a54894909ae5571b4dcaf18fa` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:32.838870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2d6d16a54894909ae5571b4dcaf18fa)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:32.840874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:32.992088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29879975-80f4-4bd6-ae76-567d72653934 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:33.004888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:33.005254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.005587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.008241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] PATH_INFO: `/v3/projects/d1e90a027f97405bb3b955ed2c51b109` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.011096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1e90a027f97405bb3b955ed2c51b109)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.019963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.055390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2d6d16a54894909ae5571b4dcaf18fa', 'request_id': 'req-01d1f0e6-9220-43f2-998e-9e83f0872856'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.118855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2d6d16a54894909ae5571b4dcaf18fa', 'request_id': 'req-01d1f0e6-9220-43f2-998e-9e83f0872856'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.140748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2d6d16a54894909ae5571b4dcaf18fa', 'request_id': 'req-01d1f0e6-9220-43f2-998e-9e83f0872856'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.172187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] The token cache is being invalidate because project c2d6d16a54894909ae5571b4dcaf18fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:33.175683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.176784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.183861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 827/1661] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:32 2022] DELETE /identity/v3/projects/c2d6d16a54894909ae5571b4dcaf18fa => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:33.185136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:33.188433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1e90a027f97405bb3b955ed2c51b109', 'request_id': 'req-479b8e97-f4ef-4f65-b744-6cab71dfdbed'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.192728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.197187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1e90a027f97405bb3b955ed2c51b109', 'request_id': 'req-479b8e97-f4ef-4f65-b744-6cab71dfdbed'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.206321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1e90a027f97405bb3b955ed2c51b109', 'request_id': 'req-479b8e97-f4ef-4f65-b744-6cab71dfdbed'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.232091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] The token cache is being invalidate because project d1e90a027f97405bb3b955ed2c51b109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:33.232490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.233925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-479b8e97-f4ef-4f65-b744-6cab71dfdbed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.241195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 835/1662] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:32 2022] DELETE /identity/v3/projects/d1e90a027f97405bb3b955ed2c51b109 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:33.244458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf48de4-bbeb-4bb5-832c-5b2abd2a529d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.244790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf48de4-bbeb-4bb5-832c-5b2abd2a529d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.245093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf48de4-bbeb-4bb5-832c-5b2abd2a529d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.273573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-cbf48de4-bbeb-4bb5-832c-5b2abd2a529d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:33.275185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-cbf48de4-bbeb-4bb5-832c-5b2abd2a529d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.382377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:33.422236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 836/1663] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:33.436719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.437378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.443676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.485081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:33.486782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.495407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1f0e6-9220-43f2-998e-9e83f0872856 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:33.501588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 837/1664] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:33.503470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:33.505002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.519735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:33.520853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.521199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.521992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] PATH_INFO: `/v3/users/4bd3e16f20c9436fb5f2da5ffd4b4233` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.525125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4bd3e16f20c9436fb5f2da5ffd4b4233)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.525775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a454d98-dba2-48e8-bfc4-5c36dd0ba2ab None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:33.527398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.532849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:33.533830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.534145 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.534477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] PATH_INFO: `/v3/projects/e0bd6eef0ea2465d8e65de4bd002c3a3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.535775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0bd6eef0ea2465d8e65de4bd002c3a3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:33.538174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:33.552754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:33.707343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0bd6eef0ea2465d8e65de4bd002c3a3', 'request_id': 'req-716cfbe9-b8bb-4e26-86e2-192ebc361d17'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.710522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4bd3e16f20c9436fb5f2da5ffd4b4233'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.712199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4bd3e16f20c9436fb5f2da5ffd4b4233'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.714288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4bd3e16f20c9436fb5f2da5ffd4b4233'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.730836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0bd6eef0ea2465d8e65de4bd002c3a3', 'request_id': 'req-716cfbe9-b8bb-4e26-86e2-192ebc361d17'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.738077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4bd3e16f20c9436fb5f2da5ffd4b4233'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.745670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0bd6eef0ea2465d8e65de4bd002c3a3', 'request_id': 'req-716cfbe9-b8bb-4e26-86e2-192ebc361d17'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.763403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-03bc410b-ff0f-466e-a695-3b841ba46ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4bd3e16f20c9436fb5f2da5ffd4b4233'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.775708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] The token cache is being invalidate because project e0bd6eef0ea2465d8e65de4bd002c3a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:33.775708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.781587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:33.801537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 838/1665] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:33 2022] DELETE /identity/v3/projects/e0bd6eef0ea2465d8e65de4bd002c3a3 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:33.801537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:33.808548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:33.813250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 828/1666] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:33 2022] DELETE /identity/v3/users/4bd3e16f20c9436fb5f2da5ffd4b4233 => generated 0 bytes in 628 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:33.816855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242524e-7c5b-44cd-8557-86803380852c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:33.822483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242524e-7c5b-44cd-8557-86803380852c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:33.823723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6242524e-7c5b-44cd-8557-86803380852c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:33.881324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6242524e-7c5b-44cd-8557-86803380852c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:33.883055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6242524e-7c5b-44cd-8557-86803380852c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.064228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716cfbe9-b8bb-4e26-86e2-192ebc361d17 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.082867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 829/1667] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:34.094571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.095002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.095410 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.102067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.116556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.117059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.117545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.118770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:34.123749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.130981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.181740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:34.183006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:34.195664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 839/1668] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:34.198032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.218383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 830/1669] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:34.224243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.238984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2067779-2d89-4154-9deb-b9cfba0b368a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.250315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb6c38-cefe-468f-afcb-8820bddcbc21 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.263757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.264397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.264673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.264940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.265652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:34.265892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.272831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.277158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.278309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.278782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.281467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.284261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.288601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.320343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 831/1670] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:36:34.326581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.352780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d4e4-f4cc-44f2-a380-f309c0011711 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.360744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.361370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.361647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.361920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.362829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.364782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.436111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 840/1671] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/projects => generated 367 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.440779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.464512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 832/1672] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/projects => generated 387 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.469076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.571953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:34.608927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:34.620980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee01b1c-90f5-4c5d-8e74-c13d53cd5701 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.633706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.635006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.635969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.636670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.638952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.643730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.644272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3134c052-4cf4-48fb-8dc1-a8077f65ccc6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.658002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.659086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.659506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.659917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.660802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.664872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.699532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 841/1673] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/users => generated 284 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.702076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 833/1674] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:34 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:34.702684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.705286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.729078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579f7db0-2732-428d-a85a-fda25b5fcece admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.739540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.741218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.741798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.742264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.743753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.746363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9065324e-13d4-44bc-bcf8-ce2f790d88ef admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.748051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.763262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.764106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.764384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.764646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.765305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.766973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.775810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 842/1675] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:34.777680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.787439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 834/1676] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:34.799262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.812511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1974d8b6-3ebf-4d7b-ab03-3c12a341d9d1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.814773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f22355-7660-4116-b5c2-c893e054b87d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.827328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.827775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.828147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.828514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.828626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.828857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.829282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.829382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.830051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.830681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.835020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.835176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.859617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 835/1677] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:34.864945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.871216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 843/1678] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:34 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:34.872190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:34.886095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e59169a-43ce-4fbd-9f5b-630d8743d185 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.906056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0830c5-738b-4d23-b72f-fea695187e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.906392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c22080-e3f7-48ad-ba27-f2754d16cb5d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:34.907098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0830c5-738b-4d23-b72f-fea695187e0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.907531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0830c5-738b-4d23-b72f-fea695187e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.907949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0830c5-738b-4d23-b72f-fea695187e0c admin admin] PATH_INFO: `/v3/projects/9784ce526b8a45858b49ecea4d086bb5/users/9892febc4aac4a10a1bed075d7a95950/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.915386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d689a9-eaaf-4124-9e8e-9691e992bb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:34.916304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d689a9-eaaf-4124-9e8e-9691e992bb06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:34.916702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d689a9-eaaf-4124-9e8e-9691e992bb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:34.916988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d689a9-eaaf-4124-9e8e-9691e992bb06 admin admin] PATH_INFO: `/v3/projects/a9a3022955ff4bc6b37374f3a9f94028/users/3d07072390f2438994255d43a89b7316/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:34.959764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0830c5-738b-4d23-b72f-fea695187e0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9784ce526b8a45858b49ecea4d086bb5, user_id=9892febc4aac4a10a1bed075d7a95950, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.967437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0830c5-738b-4d23-b72f-fea695187e0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:34.968200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d689a9-eaaf-4124-9e8e-9691e992bb06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a3022955ff4bc6b37374f3a9f94028, user_id=3d07072390f2438994255d43a89b7316, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:34.973026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d689a9-eaaf-4124-9e8e-9691e992bb06 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.025992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 844/1680] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:34 2022] PUT /identity/v3/projects/a9a3022955ff4bc6b37374f3a9f94028/users/3d07072390f2438994255d43a89b7316/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:35.029841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.031087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.032036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.033514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 836/1680] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:34 2022] PUT /identity/v3/projects/9784ce526b8a45858b49ecea4d086bb5/users/9892febc4aac4a10a1bed075d7a95950/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:35.035236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.035710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.035985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.074326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] MFA Rules not processed for user `3d07072390f2438994255d43a89b7316`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:35.075818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.150160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] MFA Rules not processed for user `9892febc4aac4a10a1bed075d7a95950`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:35.152132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:35.314513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 845/1681] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:35.315651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.432182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 837/1682] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:35.436283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.540260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcb68d9-9005-4aad-858d-fb46f57385bd None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.551602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.555341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.556002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.557534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.558636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.564582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.617748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:35.641267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 846/1683] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/projects => generated 365 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:35.643584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.683934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6faabea-fef8-49d5-a262-14be16f9b230 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.743413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:35.830679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf952874-ff50-402c-a8ad-7b3decbc74d5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:35.842909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.843295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:35.844630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.844745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:35.844858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.844935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:35.845037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.845118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:35.845194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.845732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:35.852279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.853666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:35.909554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 847/1684] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/users => generated 289 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:35.911205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.913966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 838/1685] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/projects => generated 387 bytes in 481 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:35.918052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:35.933256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32abc1d1-4063-49ee-85c9-314feb869dfd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.089518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082b274-ac03-4751-a27f-e6784b47c2fa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.115013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.115699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.115979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.116248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.116424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.117692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.117876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.118168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.118682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.119781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.120084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.123490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.168241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 848/1686] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:36.169998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.185278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 839/1687] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:35 2022] POST /identity/v3/users => generated 294 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:36.189326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.198306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893fd360-4d79-4775-b882-1b85bf1d5b91 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.209664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f8c76e-0233-4591-a404-8abed5e6dfd6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.213901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.214634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.214913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.215187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] PATH_INFO: `/v3/projects/7776f0df14d246e491d73359e527ad38/users/5549786904fc4056af715919ed815b97/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.219162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.219714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.220010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.220283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.221190 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.223760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.241663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 840/1688] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:36.251441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.256379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7776f0df14d246e491d73359e527ad38, user_id=5549786904fc4056af715919ed815b97, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.263327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.280766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a7dd90-fdcc-447d-a7df-8776a644a4af admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.295857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 849/1689] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:36 2022] PUT /identity/v3/projects/7776f0df14d246e491d73359e527ad38/users/5549786904fc4056af715919ed815b97/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.298042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.441942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.447155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.447644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.448118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.449263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.458577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.502680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 841/1690] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:36.502680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.504703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:36.509093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4dcf2-1d0f-4614-8eba-c25312be19b0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.525915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.526797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.527329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.527685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.528975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.533904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.543529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:36.545635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39816f43-5025-4568-8b43-f156be0cebfa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.563310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.563971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.564256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.564529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] PATH_INFO: `/v3/projects/61f5708c8ba34217914d0b8f7c7ac712` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.565731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=61f5708c8ba34217914d0b8f7c7ac712)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.567704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.573840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 850/1691] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:36.575643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.600923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8338d4b2-735f-4337-9304-6bab4217e318 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:36.619654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:36.620487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.620766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.621037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] PATH_INFO: `/v3/projects/405e982186214adaaf4bc0836213964c/users/3a078ca259664613bc0e01bc99f4d71f/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.641212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:36.667324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] RBAC: Authorizing `identity:create_grant(project_id=405e982186214adaaf4bc0836213964c, user_id=3a078ca259664613bc0e01bc99f4d71f, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:36.673213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:36.705862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 851/1692] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:36 2022] PUT /identity/v3/projects/405e982186214adaaf4bc0836213964c/users/3a078ca259664613bc0e01bc99f4d71f/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.715247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:36.723995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61f5708c8ba34217914d0b8f7c7ac712', 'request_id': 'req-7b004839-8ef8-4192-a01d-aef5f65788eb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:36.736219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61f5708c8ba34217914d0b8f7c7ac712', 'request_id': 'req-7b004839-8ef8-4192-a01d-aef5f65788eb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:36.758639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61f5708c8ba34217914d0b8f7c7ac712', 'request_id': 'req-7b004839-8ef8-4192-a01d-aef5f65788eb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:36.802595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] The token cache is being invalidate because project 61f5708c8ba34217914d0b8f7c7ac712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:36.803095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:36.805548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7b004839-8ef8-4192-a01d-aef5f65788eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:36.810940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 842/1693] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:36 2022] DELETE /identity/v3/projects/61f5708c8ba34217914d0b8f7c7ac712 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:36.822914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de529-d006-4187-9efe-124c10bc1594 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:36.823498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de529-d006-4187-9efe-124c10bc1594 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:36.823928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921de529-d006-4187-9efe-124c10bc1594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:36.887496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-921de529-d006-4187-9efe-124c10bc1594 None None] MFA Rules not processed for user `3a078ca259664613bc0e01bc99f4d71f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:36.890051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-921de529-d006-4187-9efe-124c10bc1594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:36.998293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8da7e-e7ad-4f58-8ef1-173c1a8e4614 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.064117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 843/1694] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:37.065572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.065866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.066147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.112163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:37.114630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:37.204897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 844/1695] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:37.206257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.208128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:37.227033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74befd-171e-494f-a71d-147a27cfd699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.227836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74befd-171e-494f-a71d-147a27cfd699 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.228230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74befd-171e-494f-a71d-147a27cfd699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.228719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74befd-171e-494f-a71d-147a27cfd699 admin admin] PATH_INFO: `/v3/projects/7776f0df14d246e491d73359e527ad38/users/5549786904fc4056af715919ed815b97/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.232933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74befd-171e-494f-a71d-147a27cfd699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7776f0df14d246e491d73359e527ad38, user_id=5549786904fc4056af715919ed815b97, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.238318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f74befd-171e-494f-a71d-147a27cfd699 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.245248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d3484a-4d55-4eff-bac9-5ae58fb7e7b9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.256290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327c3dec-c841-4a9a-96b3-8f3043b1cc41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.257977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3dec-c841-4a9a-96b3-8f3043b1cc41 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.258444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3dec-c841-4a9a-96b3-8f3043b1cc41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.258756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327c3dec-c841-4a9a-96b3-8f3043b1cc41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.259791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c3dec-c841-4a9a-96b3-8f3043b1cc41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.263696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327c3dec-c841-4a9a-96b3-8f3043b1cc41 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.270311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 852/1696] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:36 2022] PUT /identity/v3/projects/7776f0df14d246e491d73359e527ad38/users/5549786904fc4056af715919ed815b97/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:37.283124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.283124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.283124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.313756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:37.315614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:37.343699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 845/1697] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:37.345761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.346555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.347102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.388523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:37.392898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] MFA Rules not processed for user `5549786904fc4056af715919ed815b97`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:37.398012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:37.508082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 853/1698] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:37.509103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.527257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d598214-6020-49f9-905e-62e4b5130a74 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.554222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:37.639297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 846/1699] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:37.642244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.674439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.675481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.675761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.676033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.676682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.680050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.732618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 854/1700] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/users => generated 299 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:37.733620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.738313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:37.863825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b33ee9d-aa3b-4f19-b2f2-5c9a9003a817 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.878500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.879142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.879419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.879696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.880465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:37.880772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.882548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.894172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51899e-1bfc-49b8-a810-98bba6b28f5e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.903806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.907892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.908239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.908520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.914514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 847/1701] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:37.914514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.939223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9745b-2061-4c01-a4f7-3667d3168540 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:37.943505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.946876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.950866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:37.951444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:37.951726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:37.951998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:37.954783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:37.958643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:37.984695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 855/1702] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:37.988795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 848/1703] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:37.989991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:37.994121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.013233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e2ad49-cabd-475f-a6f1-82fc8f6048ba admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.026032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ff2f4c-cc2f-41a9-aefb-d72ea8a4b117 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.028712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.029345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.029627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.029903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.035251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.037766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.044912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.047676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.048039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.048388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] PATH_INFO: `/v3/projects/1f8d6d7b96264e1fbac7ac9bbf30c733/users/04a34d48279c4348af29b4f65ae00e25/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.097874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8d6d7b96264e1fbac7ac9bbf30c733, user_id=04a34d48279c4348af29b4f65ae00e25, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.101584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.117931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 849/1704] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:37 2022] POST /identity/v3/projects => generated 369 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:38.119443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.133114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 856/1705] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:37 2022] PUT /identity/v3/projects/1f8d6d7b96264e1fbac7ac9bbf30c733/users/04a34d48279c4348af29b4f65ae00e25/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:38.134450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.312125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7fd2dc-6c75-4472-a8a8-34e5bb31c4c4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.320196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:38.407593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbfacbd-2532-43d8-a2b7-110bccfef060 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.439048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.439692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.439971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.440237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.440895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.449727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.471996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.473159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.473672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.474824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.475915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.479062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.512908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 857/1706] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:38.516943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.525669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 850/1707] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:38 2022] POST /identity/v3/users => generated 285 bytes in 407 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:38.531152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.551114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42f352-acad-4b53-a8a9-d227bb7cce98 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.564343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ade74f2-b3c4-47dc-aae7-75938418be84 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.572519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f20a0-c731-4040-bebd-1ef7931d5f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.573211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f20a0-c731-4040-bebd-1ef7931d5f33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.578775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f20a0-c731-4040-bebd-1ef7931d5f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.578775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f20a0-c731-4040-bebd-1ef7931d5f33 admin admin] PATH_INFO: `/v3/projects/1f8d6d7b96264e1fbac7ac9bbf30c733/users/04a34d48279c4348af29b4f65ae00e25/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.580284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.580921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.581202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.581616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.582130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.583205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f20a0-c731-4040-bebd-1ef7931d5f33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f8d6d7b96264e1fbac7ac9bbf30c733, user_id=04a34d48279c4348af29b4f65ae00e25, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.585560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.591189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f20a0-c731-4040-bebd-1ef7931d5f33 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.616745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 851/1708] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:36:38.621954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:38.640596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 858/1709] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:38 2022] PUT /identity/v3/projects/1f8d6d7b96264e1fbac7ac9bbf30c733/users/04a34d48279c4348af29b4f65ae00e25/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:38.657418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.657903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.658328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.661067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe079b4-2306-4230-b5b6-7749e6db7270 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:38.696132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:38.757703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] MFA Rules not processed for user `04a34d48279c4348af29b4f65ae00e25`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:38.760032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:38.926038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:38.929192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:38.929491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:38.930867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:38.931625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:38.933282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:38.974961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 852/1710] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:38 2022] GET /identity/v3/roles => generated 1420 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:38.984524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:39.019988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ef02bd-f10f-4d3b-941a-0d1cedb07fab admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:39.034294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 859/1711] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:38 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:39.051169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:39.069361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b32676-1fc3-484d-ac33-acf7d979a038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:39.070248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32676-1fc3-484d-ac33-acf7d979a038 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.075493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32676-1fc3-484d-ac33-acf7d979a038 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.076060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b32676-1fc3-484d-ac33-acf7d979a038 admin admin] PATH_INFO: `/v3/projects/6c9b1fe76e3e42679e0f777f1647c011/users/4f4c98a691db4d54867f3b264e2a80fa/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.138751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b32676-1fc3-484d-ac33-acf7d979a038 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c9b1fe76e3e42679e0f777f1647c011, user_id=4f4c98a691db4d54867f3b264e2a80fa, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:39.144947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b32676-1fc3-484d-ac33-acf7d979a038 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:39.214217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 853/1712] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:38 2022] PUT /identity/v3/projects/6c9b1fe76e3e42679e0f777f1647c011/users/4f4c98a691db4d54867f3b264e2a80fa/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:39.214964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8b805-496a-4d6a-988e-2ea90b903ae3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:39.216628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.216858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.217135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.280878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] MFA Rules not processed for user `4f4c98a691db4d54867f3b264e2a80fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:39.282961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:39.389512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:39.398120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:39.398551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.401752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.402046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.468742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 854/1713] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:39.469896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:39.594823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:39.603449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:39.604948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:39.637404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 860/1714] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 587 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:39.749667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a607-65b2-4c7e-949a-2dd45ec856a4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:39.778699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:39.780372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:39.780659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:39.785344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:39.786154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:39.797946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:39.902160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 855/1715] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:39 2022] POST /identity/v3/projects => generated 369 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:39.903868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:39.919743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f880f8c8-b3b8-4fc9-96d1-eb4e11adc236 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.082167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.084550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:40.119466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.121097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.121381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.121659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.122368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.136607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.211346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 861/1716] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:39 2022] POST /identity/v3/users => generated 291 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:40.214022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.235733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de0103f-f6c9-42ad-9bdd-59f30acd55fe admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.248745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.249527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.249813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.251392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.252156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.257496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.296415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 862/1717] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:40.297901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.299253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243a8e9a-999e-445e-8861-4eff90ffbcfd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.313986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.314796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.315075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.315347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] PATH_INFO: `/v3/users/d7a8af6c549841a7a2f690fb2f642b7a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.316770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7a8af6c549841a7a2f690fb2f642b7a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.321108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59a3f25-3f71-4692-a86f-8a57b27c16b5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.324141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.334712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.335683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.335974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.336326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] PATH_INFO: `/v3/projects/a413b5b68b4041388f0877ab13091172/users/89a563fefe9947f091551bc2b2198456/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.379024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a413b5b68b4041388f0877ab13091172, user_id=89a563fefe9947f091551bc2b2198456, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.382200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.421124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 863/1718] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:40 2022] PUT /identity/v3/projects/a413b5b68b4041388f0877ab13091172/users/89a563fefe9947f091551bc2b2198456/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:40.423154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.462955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7a8af6c549841a7a2f690fb2f642b7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:40.463660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7a8af6c549841a7a2f690fb2f642b7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:40.464467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7a8af6c549841a7a2f690fb2f642b7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:40.489116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7a8af6c549841a7a2f690fb2f642b7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:40.506927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1d5ebfc3-05fa-42dc-8fbd-bb283e6ba978 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7a8af6c549841a7a2f690fb2f642b7a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:40.525661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:40.582788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 856/1719] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:40 2022] DELETE /identity/v3/users/d7a8af6c549841a7a2f690fb2f642b7a => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:40.589193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.589297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.589642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.597742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2bb88f-598a-4230-9930-e362a4e7bd40 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.664318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:40.670130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:40.783970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:40.813836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.814619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.815005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.815496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.816580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.818913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.860885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 864/1720] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:40 2022] GET /identity/v3/roles => generated 1420 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:40.861924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.871437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:40.875520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 857/1721] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:40.909977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3507b838-6151-4142-b530-724c146f9d83 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:40.927995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62e712-2a2a-4804-ab67-1ab6d06028a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:40.928837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62e712-2a2a-4804-ab67-1ab6d06028a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:40.929539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62e712-2a2a-4804-ab67-1ab6d06028a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:40.930052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62e712-2a2a-4804-ab67-1ab6d06028a6 admin admin] PATH_INFO: `/v3/projects/a413b5b68b4041388f0877ab13091172/users/89a563fefe9947f091551bc2b2198456/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:40.935277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62e712-2a2a-4804-ab67-1ab6d06028a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a413b5b68b4041388f0877ab13091172, user_id=89a563fefe9947f091551bc2b2198456, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:40.941297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62e712-2a2a-4804-ab67-1ab6d06028a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:40.982813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:40.984321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:40.988919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 865/1722] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:40 2022] PUT /identity/v3/projects/a413b5b68b4041388f0877ab13091172/users/89a563fefe9947f091551bc2b2198456/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:40.999899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:41.000194 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:41.000474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:41.132825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] MFA Rules not processed for user `89a563fefe9947f091551bc2b2198456`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:41.134701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:41.223221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084f3765-ba04-4159-b2ef-30f1ef0ff4ef None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:41.370843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:41.372556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:41.374673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:41.375970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:41.461951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 866/1723] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:41.468527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:41.471478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:41.633290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:41.638938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:41.643801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:41.666626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 858/1724] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 684 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:41.668602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:41.671788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:41.704340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d4d770-2e4a-4548-9263-2c6e7e2a4503 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:41.709562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:41.710885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:41.711290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:41.711726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:41.715032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:41.724983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5e2eab-ac3e-455e-b588-872c48128412 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:41.754382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:41.760741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:41.763187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:41.763694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:41.797159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:41.799610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:41.839764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 859/1725] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:41.945616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:41.947568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e575324-9aec-487b-b154-4b20c2a4187c service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:42.003778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 867/1726] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:41 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:42.079889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:42.094114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f4216-058c-4b2a-b52b-65c00220d471 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:42.101683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:42.102827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:42.103379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:42.103913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:42.108033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:42.227150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:42.228810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:42.253041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 860/1727] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:42 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:42.611961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:42.612529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:42.612829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:42.697016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] MFA Rules not processed for user `0b294c0072b740cc960bc53e1f806ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:42.702620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:42.760529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:42.870582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:42.871687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:42.929179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287250ae-bfd6-45f4-a0e9-b5d4f71a4e98 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:42.960513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:42.962522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:42.963503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:42.964774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] PATH_INFO: `/v3/projects/1ce5ae57e2664ca1a071f3a148c0cfb1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:42.974910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ce5ae57e2664ca1a071f3a148c0cfb1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:42.989432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:43.017630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 868/1728] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 407 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:43.044597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:43.046023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.184660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ce5ae57e2664ca1a071f3a148c0cfb1', 'request_id': 'req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.195651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ce5ae57e2664ca1a071f3a148c0cfb1', 'request_id': 'req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.219506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ce5ae57e2664ca1a071f3a148c0cfb1', 'request_id': 'req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.244051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] The token cache is being invalidate because project 1ce5ae57e2664ca1a071f3a148c0cfb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:43.245098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.246472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cdf5c0e6-a60a-4d6e-a81d-adc0adba5e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.249014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 861/1729] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:42 2022] DELETE /identity/v3/projects/1ce5ae57e2664ca1a071f3a148c0cfb1 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:43.256221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.256611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.256991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.273763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3daf34-43cf-442d-82be-6f1d074b008d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:43.322002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:43.323945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.377072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:43.378264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.379752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.380221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.382344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.425484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 862/1730] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:43.435002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:43.436768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.467309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37657a59-9288-4a17-a5aa-a2630327a774 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:43.477558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:43.479129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.479719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.480144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] PATH_INFO: `/v3/users/3d07072390f2438994255d43a89b7316` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.482300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d07072390f2438994255d43a89b7316)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:43.484888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:43.489749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:43.506488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:43.523603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:43.525243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc1b58-2b5f-4f60-82b2-146c23bac625 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:43.537504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 869/1731] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:36:43.539518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.539828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.540260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.568390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:43.569965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.588852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 870/1732] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:43.591265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:43.610009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fcd181-44e8-4fe1-83c1-07fdf73f0dd7 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:43.622893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:43.624479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.627228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.627495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.628581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:43.630079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:43.641974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:43.642206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d07072390f2438994255d43a89b7316'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.642966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d07072390f2438994255d43a89b7316'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.643649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d07072390f2438994255d43a89b7316'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.666579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d07072390f2438994255d43a89b7316'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.669584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 871/1733] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:43.674774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:43.677459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:43.686706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0fa39f41-d34d-4198-9674-56315c736263 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d07072390f2438994255d43a89b7316'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:43.704273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7982a-6bf2-4280-bc19-f7acf5e40aa3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:43.719041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:43.719756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.719960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.720237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.721687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:43.726933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:43.734786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 863/1734] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:43 2022] DELETE /identity/v3/users/3d07072390f2438994255d43a89b7316 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:43.740807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:43.741097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:43.741378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:43.810729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 872/1735] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:43 2022] POST /identity/v3/projects => generated 403 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:43.814973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:43.820595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:43.822074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:44.041226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 864/1736] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:44.104162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0ba446-bbdd-49ff-a6b8-9e409ef6acf8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.127759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.128187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.128475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.129244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.130655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.136410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.178773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.181495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:44.207364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 873/1737] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:43 2022] POST /identity/v3/users => generated 302 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:36:44.208939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.275921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b8619e-ae19-401f-b058-4f89694d3d82 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.301273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.307079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.310382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.310382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.312769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.317681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.359843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 874/1738] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:44.368727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.404184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4145a9af-af1c-41cf-9d85-fe3425ba77a6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.408783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617d8a-87ff-454a-be92-68c55a391572 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.453619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.453914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.454220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.454498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.455744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.461759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdf663-d163-4104-bb55-7000a43193d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.462769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf663-d163-4104-bb55-7000a43193d5 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.463025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf663-d163-4104-bb55-7000a43193d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.464600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf663-d163-4104-bb55-7000a43193d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.472986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.572029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdf663-d163-4104-bb55-7000a43193d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.581734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdf663-d163-4104-bb55-7000a43193d5 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.589660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 875/1739] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:44 2022] GET /identity/v3/roles => generated 1420 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:44.594140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:44.662428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 865/1740] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:44.669642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8122504-1778-4cdf-b180-8fe2d1b5dfd9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:44.684936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:44.685876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.686348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.686776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] PATH_INFO: `/v3/projects/d270caa6372a41c59350c486d2de18c8/users/25bee7c69d074709a2ffa3ffbe7cd864/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.744830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d270caa6372a41c59350c486d2de18c8, user_id=25bee7c69d074709a2ffa3ffbe7cd864, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:44.762000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:44.851376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 876/1741] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:44 2022] PUT /identity/v3/projects/d270caa6372a41c59350c486d2de18c8/users/25bee7c69d074709a2ffa3ffbe7cd864/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:44.858367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4684-a36b-4d88-9256-873d6075f496 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:44.859767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4684-a36b-4d88-9256-873d6075f496 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:44.860718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ed4684-a36b-4d88-9256-873d6075f496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:44.935814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-24ed4684-a36b-4d88-9256-873d6075f496 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:44.942280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-24ed4684-a36b-4d88-9256-873d6075f496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:45.003254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:45.217474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 866/1742] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:45.284517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:45.290123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:45.603323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:45.614309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f76b8d-4258-450c-b196-67a3f1c89609 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:45.619261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:45.620056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:45.620448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:45.620833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:45.653765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:45.658933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:45.721076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 877/1743] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:46.430492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:46.431037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:46.431144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:46.606694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] MFA Rules not processed for user `b1d9d5fd0db54ddf85e05c77dd7f6c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:46.608888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:46.930729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 867/1744] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:46 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 502 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:46.947193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:46.949874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:47.379242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab45c4d-0d33-450c-a471-139a30212846 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:47.404329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:47.404329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:47.404329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:47.404329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:47.475335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:47.477157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:47.521695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 878/1745] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 568 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:48.348175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:48.350178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:48.465178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:48.467267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:48.493595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c230f39-77ef-4180-85a4-561996d5b623 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:48.494980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bafe83-e4ef-4528-891c-df35a75070ec service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:48.499472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:48.500340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.500987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.501382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.509349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:48.510530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.512219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.512981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:48.513492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] PATH_INFO: `/v3/users/4f4c98a691db4d54867f3b264e2a80fa` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.527134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f4c98a691db4d54867f3b264e2a80fa)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:48.529529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:48.630701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f4c98a691db4d54867f3b264e2a80fa'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:48.632114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f4c98a691db4d54867f3b264e2a80fa'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:48.632900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f4c98a691db4d54867f3b264e2a80fa'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:48.658678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f4c98a691db4d54867f3b264e2a80fa'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:48.691725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:48.696069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0e578-6872-4467-b65b-e4730a7dcebd service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:48.710091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a5e5093a-b93c-4d8c-aafc-60154d113846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f4c98a691db4d54867f3b264e2a80fa'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:48.730753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 868/1746] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:48 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:48.736049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.736899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.738177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.745064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 879/1747] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:48 2022] DELETE /identity/v3/users/4f4c98a691db4d54867f3b264e2a80fa => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:36:48.749430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:48.749927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:48.752639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:48.797184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:48.804459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:48.810903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:48.870708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] MFA Rules not processed for user `7f1047b63c4148ca8073e3414ef84d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:48.878862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:48.886851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:48.948007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 880/1748] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:48.948007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:48.950761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:49.003507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaa0c20-fc14-496b-b941-72894071c0de None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:49.016979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:49.019380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:49.019727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:49.022460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] PATH_INFO: `/v3/projects/a9a3022955ff4bc6b37374f3a9f94028` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:49.028638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a3022955ff4bc6b37374f3a9f94028)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:49.040651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:49.160968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 869/1749] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:48 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 430 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:49.166197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:49.168051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:49.270061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a3022955ff4bc6b37374f3a9f94028', 'request_id': 'req-570aa7f5-ee65-47a4-8739-7f5bc135b083'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:49.297410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a3022955ff4bc6b37374f3a9f94028', 'request_id': 'req-570aa7f5-ee65-47a4-8739-7f5bc135b083'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:49.332537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a3022955ff4bc6b37374f3a9f94028', 'request_id': 'req-570aa7f5-ee65-47a4-8739-7f5bc135b083'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:49.359712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] The token cache is being invalidate because project a9a3022955ff4bc6b37374f3a9f94028 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:49.360006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:49.361899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-570aa7f5-ee65-47a4-8739-7f5bc135b083 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:49.366445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 881/1750] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:48 2022] DELETE /identity/v3/projects/a9a3022955ff4bc6b37374f3a9f94028 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:49.367773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:49.371598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:49.371598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:49.399206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4f5715-0aae-49f0-9cac-8504ed1fff65 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:49.427573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] MFA Rules not processed for user `7f1047b63c4148ca8073e3414ef84d28`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:49.433496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:49.620270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:49.623400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:49.623961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:49.624446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:49.626946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:49.677440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 882/1751] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:49.679260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:49.684864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:49.811842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:49.813656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:49.862211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 870/1752] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 700 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:49.863625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:49.866488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:49.877310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:49.910988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc1e1d-072a-44e4-bab2-536f0ad7df28 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:49.937163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:49.937825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:49.938101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:49.945951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] PATH_INFO: `/v3/users/5549786904fc4056af715919ed815b97` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:49.950651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5549786904fc4056af715919ed815b97)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:49.957100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:50.004158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:50.103587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e17ec50-e24e-4c28-9c4e-3d7166b58966 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:50.111964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5549786904fc4056af715919ed815b97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:50.113091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5549786904fc4056af715919ed815b97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:50.116811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5549786904fc4056af715919ed815b97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:50.134682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5549786904fc4056af715919ed815b97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:50.159007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:50.159007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:50.159007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:50.159007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:50.163120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:50.180794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-18f46b23-9af4-4fcf-9513-9e84952c10c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5549786904fc4056af715919ed815b97'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:50.197348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:50.200669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a804dd6d-7219-4707-a2d1-67f5f68832d7 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:50.221953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 883/1753] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:49 2022] DELETE /identity/v3/users/5549786904fc4056af715919ed815b97 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:50.224209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fcb39-11b5-4502-9ee9-0a58ee1b4258 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:50.224496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fcb39-11b5-4502-9ee9-0a58ee1b4258 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:50.224770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7fcb39-11b5-4502-9ee9-0a58ee1b4258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:50.232309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 871/1754] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:49 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:50.422035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bd7fcb39-11b5-4502-9ee9-0a58ee1b4258 None None] MFA Rules not processed for user `76c0df49fa9342778a56eae098b2f2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:50.423321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bd7fcb39-11b5-4502-9ee9-0a58ee1b4258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:50.662648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:50.663592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:50.664044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:50.667076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 884/1755] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:50.668596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:50.668883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:50.669156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:50.718701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] MFA Rules not processed for user `76c0df49fa9342778a56eae098b2f2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:50.719958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:50.757370 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] MFA Rules not processed for user `25bee7c69d074709a2ffa3ffbe7cd864`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:50.758897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:50.762626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 885/1756] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:50.763822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:50.767636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:50.916876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154f9f90-9eec-4ddb-82b7-4e505254c975 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:50.929318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6592d0-930c-4acd-931e-d38f40d3153e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:50.929956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6592d0-930c-4acd-931e-d38f40d3153e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:50.933229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6592d0-930c-4acd-931e-d38f40d3153e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:50.933614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6592d0-930c-4acd-931e-d38f40d3153e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:50.952536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6592d0-930c-4acd-931e-d38f40d3153e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:50.955702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6592d0-930c-4acd-931e-d38f40d3153e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:50.974674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 886/1757] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:51.022278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.022571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.023428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.035448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:51.083807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] MFA Rules not processed for user `0b294c0072b740cc960bc53e1f806ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:51.088335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:51.132779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 872/1758] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:50 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:51.133884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:51.139949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:51.299693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 887/1759] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:51.332049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47284d3a-6d08-4ae0-b955-f157fa26b2da None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:51.358721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:51.359349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.359630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.362878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.366517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:51.373726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:51.429434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 873/1760] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:51 2022] POST /identity/v3/projects => generated 405 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:51.433805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:51.454930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:51.459239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:51.466573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:51.484540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19096f5-a490-45b3-95f8-903d1af8577d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:51.619377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55ad99c-e0b3-49bf-a713-8ac5e4719503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:51.620467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad99c-e0b3-49bf-a713-8ac5e4719503 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.620974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad99c-e0b3-49bf-a713-8ac5e4719503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.621469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55ad99c-e0b3-49bf-a713-8ac5e4719503 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.626942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ad99c-e0b3-49bf-a713-8ac5e4719503 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:51.630809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2465eb-c65c-4e62-a5d7-2155dc6a11a7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:51.631448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55ad99c-e0b3-49bf-a713-8ac5e4719503 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:51.646611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:51.647418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.647572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.647821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] PATH_INFO: `/v3/projects/7776f0df14d246e491d73359e527ad38` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.659821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7776f0df14d246e491d73359e527ad38)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:51.671421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:51.681229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 888/1761] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:51 2022] POST /identity/v3/users => generated 309 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:51.682336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478505f-0659-4d3e-b532-78f425505ba9 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.682646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478505f-0659-4d3e-b532-78f425505ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.683050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e478505f-0659-4d3e-b532-78f425505ba9 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.684127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 889/1762] 10.222.0.91 () {54 vars in 912 bytes} [Wed Aug 24 03:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:51.686639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478505f-0659-4d3e-b532-78f425505ba9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:51.717883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e478505f-0659-4d3e-b532-78f425505ba9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:51.728947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cd0b9c-ed87-4a70-8248-aa031aca35f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:51.729731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b9c-ed87-4a70-8248-aa031aca35f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.729838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b9c-ed87-4a70-8248-aa031aca35f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.730089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cd0b9c-ed87-4a70-8248-aa031aca35f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.730785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b9c-ed87-4a70-8248-aa031aca35f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:51.732453 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cd0b9c-ed87-4a70-8248-aa031aca35f8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:51.756494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 890/1763] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:51 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:51.758028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c8686-0d21-456a-ac27-bfd2545c2bf6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.758327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c8686-0d21-456a-ac27-bfd2545c2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.758711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4c8686-0d21-456a-ac27-bfd2545c2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:51.811779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7776f0df14d246e491d73359e527ad38', 'request_id': 'req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:51.829412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7776f0df14d246e491d73359e527ad38', 'request_id': 'req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:51.839187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5b4c8686-0d21-456a-ac27-bfd2545c2bf6 None None] MFA Rules not processed for user `5ae7930c17434ae1b6df030b0599e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:51.840371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5b4c8686-0d21-456a-ac27-bfd2545c2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:51.856042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7776f0df14d246e491d73359e527ad38', 'request_id': 'req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:51.878301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] The token cache is being invalidate because project 7776f0df14d246e491d73359e527ad38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:51.878718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:51.879775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:51.881855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 874/1764] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:51 2022] DELETE /identity/v3/projects/7776f0df14d246e491d73359e527ad38 => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:51.882901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:51.885525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:51.986335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 891/1765] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:51.988249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:51.988680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:51.990344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.028672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:52.030517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.047341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8af52d-5fd4-4875-8508-fbbbf8ae2841 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.056513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 892/1766] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:52.058572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.061496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.064629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270ef7a-0d48-4370-9c3d-637b3ad38343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.065356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270ef7a-0d48-4370-9c3d-637b3ad38343 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.065649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270ef7a-0d48-4370-9c3d-637b3ad38343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.065931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270ef7a-0d48-4370-9c3d-637b3ad38343 admin admin] PATH_INFO: `/v3/projects/5374bd1ac87b41dcb313cac4e190ca88/users/c6b2dd658b3f4f49ad5ca777b6dff8b1/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.111304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229fd434-cac6-4fa9-852b-3c7c2199b939 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.121013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270ef7a-0d48-4370-9c3d-637b3ad38343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5374bd1ac87b41dcb313cac4e190ca88, user_id=c6b2dd658b3f4f49ad5ca777b6dff8b1, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.124308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.125748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.126001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270ef7a-0d48-4370-9c3d-637b3ad38343 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.127804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.128390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.153107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 875/1767] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:51 2022] PUT /identity/v3/projects/5374bd1ac87b41dcb313cac4e190ca88/users/c6b2dd658b3f4f49ad5ca777b6dff8b1/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:52.155048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.155499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.156079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.159839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.169878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.180538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:52.182061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.195565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:52.208632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 893/1768] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:52.210038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.215693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.247244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:52.263808 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c658a0-0e3d-4edc-beee-49572e46872d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.336434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 876/1769] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:52.338671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.365729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c8759d-a4ac-4e3c-935b-e541616cb530 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.379786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.382887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.383179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.383454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.385828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.387684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.428649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 877/1770] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:52.428649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.451758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78abc9f-ebb5-4e0e-814c-592d7ea846d0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.464418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.465048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.465325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.465598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.466404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:52.466774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.468380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.479468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.480392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.480822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.481412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] PATH_INFO: `/v3/users/9892febc4aac4a10a1bed075d7a95950` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.482972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9892febc4aac4a10a1bed075d7a95950)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.486677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.503967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 878/1771] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:52.505124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.509837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.537098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831458c3-e30a-4412-ae24-16119fc9b501 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.543197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.544097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.544377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.544654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] PATH_INFO: `/v3/projects/6c9b1fe76e3e42679e0f777f1647c011` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.546173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c9b1fe76e3e42679e0f777f1647c011)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.548473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.589908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9892febc4aac4a10a1bed075d7a95950'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.590876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9892febc4aac4a10a1bed075d7a95950'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.591801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9892febc4aac4a10a1bed075d7a95950'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.605957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9892febc4aac4a10a1bed075d7a95950'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.625891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c9b1fe76e3e42679e0f777f1647c011', 'request_id': 'req-bd00778d-d77f-4907-a30a-cac50ea67fcf'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.626476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9892febc4aac4a10a1bed075d7a95950'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.635688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c9b1fe76e3e42679e0f777f1647c011', 'request_id': 'req-bd00778d-d77f-4907-a30a-cac50ea67fcf'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.648250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c9b1fe76e3e42679e0f777f1647c011', 'request_id': 'req-bd00778d-d77f-4907-a30a-cac50ea67fcf'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.660118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 894/1772] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:52 2022] DELETE /identity/v3/users/9892febc4aac4a10a1bed075d7a95950 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:52.667749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] The token cache is being invalidate because project 6c9b1fe76e3e42679e0f777f1647c011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:52.668101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.668730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.669046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bd00778d-d77f-4907-a30a-cac50ea67fcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:52.671165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 879/1773] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:52 2022] DELETE /identity/v3/projects/6c9b1fe76e3e42679e0f777f1647c011 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:52.672918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbe6ef-3bca-436c-ab68-0957ecd6c404 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.673835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbe6ef-3bca-436c-ab68-0957ecd6c404 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.673835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bbe6ef-3bca-436c-ab68-0957ecd6c404 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.680059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.681083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 880/1774] 10.222.0.91 () {54 vars in 912 bytes} [Wed Aug 24 03:36:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:52.681960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbe6ef-3bca-436c-ab68-0957ecd6c404 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.684170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-65bbe6ef-3bca-436c-ab68-0957ecd6c404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.797617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bbe6ef-3bca-436c-ab68-0957ecd6c404 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.803931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.804757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.805340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.805884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.806714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.809101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.823361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09827ba5-c726-4382-ad84-83569cae6279 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:52.830831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e7f1a8-3542-4384-8cae-3e96f46716d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:52.831529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f1a8-3542-4384-8cae-3e96f46716d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.832342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f1a8-3542-4384-8cae-3e96f46716d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.832342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7f1a8-3542-4384-8cae-3e96f46716d0 admin admin] PATH_INFO: `/v3/projects/5374bd1ac87b41dcb313cac4e190ca88/users/c6b2dd658b3f4f49ad5ca777b6dff8b1/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.834940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7f1a8-3542-4384-8cae-3e96f46716d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5374bd1ac87b41dcb313cac4e190ca88, user_id=c6b2dd658b3f4f49ad5ca777b6dff8b1, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:52.839489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e7f1a8-3542-4384-8cae-3e96f46716d0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:52.856478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 895/1775] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:52 2022] PUT /identity/v3/projects/5374bd1ac87b41dcb313cac4e190ca88/users/c6b2dd658b3f4f49ad5ca777b6dff8b1/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:52.856478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb7dd-0b68-4f06-80ea-8e9316dfff31 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:52.856478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb7dd-0b68-4f06-80ea-8e9316dfff31 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:52.856478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb7dd-0b68-4f06-80ea-8e9316dfff31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:52.869180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:52.896112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 881/1776] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:52 2022] POST /identity/v3/projects => generated 387 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:52.896824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:52.912745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.913035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7cffb7dd-0b68-4f06-80ea-8e9316dfff31 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:52.914362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7cffb7dd-0b68-4f06-80ea-8e9316dfff31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:52.946947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:53.054750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 896/1777] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:53.056527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.056815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.057083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.080721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] MFA Rules not processed for user `5ae7930c17434ae1b6df030b0599e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:53.082334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.110749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e227e38-49df-41c6-83cd-ae4c06b2cfd2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.121099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.121931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.122234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.123530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] PATH_INFO: `/v3/users/89a563fefe9947f091551bc2b2198456` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.126229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] RBAC: Authorizing `identity:delete_user(user_id=89a563fefe9947f091551bc2b2198456)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.131620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.236546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89a563fefe9947f091551bc2b2198456'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:53.237449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89a563fefe9947f091551bc2b2198456'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:53.238540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89a563fefe9947f091551bc2b2198456'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:53.242904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 897/1778] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:53.245251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:53.260466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89a563fefe9947f091551bc2b2198456'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:53.270889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc0adf-06bd-4540-8a83-145eb4c682a6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.274259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89a563fefe9947f091551bc2b2198456'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:53.283062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.284380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.284869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.285341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.287494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.292307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.322641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 882/1779] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:52 2022] DELETE /identity/v3/users/89a563fefe9947f091551bc2b2198456 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:53.329234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:53.337611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.364470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 898/1780] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:53 2022] POST /identity/v3/users => generated 294 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:53.364470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:53.367188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.409556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f0cc21-6ffc-4c7b-aa2b-ef4a166ffa2d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.487308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958ec2a3-8735-4348-8a47-c9377ff02aad admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.497869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.499233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.500659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.501112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.507590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.563618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.564927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.565702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.566284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.570798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.595168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.601606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:53.615843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.641206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 899/1781] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:53.642908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:53.645728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:53.753787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.759343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.778437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:53.814518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba19cb1-b0d2-441c-90e7-3fde1a4f450e service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.820751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 883/1782] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:53.822656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.823896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.825159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.826607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.828438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.830621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.859184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 900/1783] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:53.865303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:53.882006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc1398-c1c4-4283-b0a4-963d8b540033 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.897154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353def3b-cf52-451e-a10a-0472003f7a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.898234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353def3b-cf52-451e-a10a-0472003f7a8d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.898730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353def3b-cf52-451e-a10a-0472003f7a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:53.899208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353def3b-cf52-451e-a10a-0472003f7a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:53.900657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353def3b-cf52-451e-a10a-0472003f7a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:53.906998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353def3b-cf52-451e-a10a-0472003f7a8d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:53.944275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 884/1784] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:53.948781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:53.980552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04485ee-e68f-46d2-b23f-89957c1591af admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:53.995035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c669ba23-8821-46b5-b2fe-9915b56c117b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:53.996487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ba23-8821-46b5-b2fe-9915b56c117b admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:53.997432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ba23-8821-46b5-b2fe-9915b56c117b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.006012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c669ba23-8821-46b5-b2fe-9915b56c117b admin admin] PATH_INFO: `/v3/projects/c2cac24d02c641be9e27258960f6b204/users/00954785920d481495ecfe9f3f0e4767/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.073314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ba23-8821-46b5-b2fe-9915b56c117b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2cac24d02c641be9e27258960f6b204, user_id=00954785920d481495ecfe9f3f0e4767, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:54.076209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c669ba23-8821-46b5-b2fe-9915b56c117b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:54.124150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 901/1785] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:53 2022] PUT /identity/v3/projects/c2cac24d02c641be9e27258960f6b204/users/00954785920d481495ecfe9f3f0e4767/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:54.136292 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.137106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.137734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.235753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] MFA Rules not processed for user `00954785920d481495ecfe9f3f0e4767`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:54.237627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:54.281636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9075f-a291-4309-bf22-101b073a422a None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.282309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9075f-a291-4309-bf22-101b073a422a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.282581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f9075f-a291-4309-bf22-101b073a422a None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.284264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 902/1786] 10.222.0.91 () {54 vars in 961 bytes} [Wed Aug 24 03:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:36:54.296907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.297263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.297439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.346982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:54.354500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:54.428439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 885/1787] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:54 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:54.432726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:54.534381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 903/1788] 10.222.0.91 () {58 vars in 1058 bytes} [Wed Aug 24 03:36:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Aug 24 03:36:54.544466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:54.560052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c240925c-63d4-40ff-9332-b8aa92ed78fb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:54.565580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:54.566235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.566519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.566794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.569123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:54.585884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:54.587817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:54.607328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 904/1789] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:54.611354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d17203-0f0f-4885-9c50-6e24eecce04c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:54.627267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:54.628081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.628364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.628634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.629282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:54.632114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:54.713340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:54.716832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 886/1790] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:54 2022] POST /identity/v3/projects => generated 387 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:54.736862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c102174-c0dc-48c7-a6e6-5c3379522074 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:54.991215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:54.992200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:54.992200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:54.992441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:54.993243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:54.995136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:55.050309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 905/1791] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:54 2022] POST /identity/v3/users => generated 300 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:55.053025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:55.130602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51e24ef-2538-4a96-b03a-e1ab535f4972 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:55.160834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac050d-5ad3-46b1-8346-7395a9e1738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:55.164906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac050d-5ad3-46b1-8346-7395a9e1738e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.166078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac050d-5ad3-46b1-8346-7395a9e1738e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.183105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac050d-5ad3-46b1-8346-7395a9e1738e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.185043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac050d-5ad3-46b1-8346-7395a9e1738e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:55.199039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac050d-5ad3-46b1-8346-7395a9e1738e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:55.257692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:55.262242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 887/1792] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:55.280407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9870007-9bc8-4fee-b97c-db8725242575 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:55.292436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:55.293117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.293402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.293680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] PATH_INFO: `/v3/projects/bcc0b46c5ff543d884811206546d4fc7/users/55884db659f84f12bc28238d85d1c818/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.353703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.354347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.354602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0b46c5ff543d884811206546d4fc7, user_id=55884db659f84f12bc28238d85d1c818, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:55.355045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.358068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:55.386511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 906/1793] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:55 2022] PUT /identity/v3/projects/bcc0b46c5ff543d884811206546d4fc7/users/55884db659f84f12bc28238d85d1c818/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:55.398321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:55.427913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] MFA Rules not processed for user `b1d9d5fd0db54ddf85e05c77dd7f6c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:55.430009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:55.453418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:55.493441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:55.657785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d39ea-c43b-4d80-9364-35baf9ba7841 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:55.697892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:55.704520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.708298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.708902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.710131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:55.735186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:55.774868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 888/1794] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 422 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:55.815275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 907/1795] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:55 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:55.817701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:55.839271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:55.843326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:55.850017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3bd60-2b23-4efa-9526-d14c13b55de1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:55.877939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dee8e-ac04-43c9-9140-de665d94481f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:55.880081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dee8e-ac04-43c9-9140-de665d94481f admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.882600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dee8e-ac04-43c9-9140-de665d94481f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.883378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dee8e-ac04-43c9-9140-de665d94481f admin admin] PATH_INFO: `/v3/projects/bcc0b46c5ff543d884811206546d4fc7/users/55884db659f84f12bc28238d85d1c818/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.888757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dee8e-ac04-43c9-9140-de665d94481f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc0b46c5ff543d884811206546d4fc7, user_id=55884db659f84f12bc28238d85d1c818, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:55.889730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050fa0ad-d739-4498-9589-3f8b7bd082b0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:55.901057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dee8e-ac04-43c9-9140-de665d94481f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:55.917847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:55.919003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.919504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.919988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] PATH_INFO: `/v3/projects/9784ce526b8a45858b49ecea4d086bb5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.921147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 889/1796] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:55 2022] PUT /identity/v3/projects/bcc0b46c5ff543d884811206546d4fc7/users/55884db659f84f12bc28238d85d1c818/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:55.928815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9784ce526b8a45858b49ecea4d086bb5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:55.933186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:55.933481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:55.933750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:55.947227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:56.030504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] MFA Rules not processed for user `55884db659f84f12bc28238d85d1c818`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:56.047663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:56.189648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9784ce526b8a45858b49ecea4d086bb5', 'request_id': 'req-df077fa4-66b9-4f02-b414-df896bb3cd15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.211385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9784ce526b8a45858b49ecea4d086bb5', 'request_id': 'req-df077fa4-66b9-4f02-b414-df896bb3cd15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.224267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9784ce526b8a45858b49ecea4d086bb5', 'request_id': 'req-df077fa4-66b9-4f02-b414-df896bb3cd15'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.251686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] The token cache is being invalidate because project 9784ce526b8a45858b49ecea4d086bb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:56.252320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.256113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.262005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 908/1797] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:55 2022] DELETE /identity/v3/projects/9784ce526b8a45858b49ecea4d086bb5 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:56.263136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:56.267239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:56.422484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df077fa4-66b9-4f02-b414-df896bb3cd15 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:56.432842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:56.433933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:56.434373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:56.434740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] PATH_INFO: `/v3/projects/a413b5b68b4041388f0877ab13091172` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:56.441624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a413b5b68b4041388f0877ab13091172)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:56.445032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:56.479601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 890/1798] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 548 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:56.481165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:56.486042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:56.510021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:56.559562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a413b5b68b4041388f0877ab13091172', 'request_id': 'req-5cf38f30-9bc2-4524-bfc4-2918a139468f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.578643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a413b5b68b4041388f0877ab13091172', 'request_id': 'req-5cf38f30-9bc2-4524-bfc4-2918a139468f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.591449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a413b5b68b4041388f0877ab13091172', 'request_id': 'req-5cf38f30-9bc2-4524-bfc4-2918a139468f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.636351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] The token cache is being invalidate because project a413b5b68b4041388f0877ab13091172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:36:56.637291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.638647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:56.643999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 909/1799] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:56 2022] DELETE /identity/v3/projects/a413b5b68b4041388f0877ab13091172 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:56.645024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:56.649924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:56.731967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c42d1fb-1522-4069-bbc0-bfcb876acc6f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:56.821376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf38f30-9bc2-4524-bfc4-2918a139468f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:56.835457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:56.836257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:56.836538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:56.836986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:56.842043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:56.847963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:56.885363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:56.888290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:56.888290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:56.888627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] PATH_INFO: `/v3/users/3a078ca259664613bc0e01bc99f4d71f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:56.892345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a078ca259664613bc0e01bc99f4d71f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:56.900545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:57.085130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a078ca259664613bc0e01bc99f4d71f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.087097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a078ca259664613bc0e01bc99f4d71f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.088071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a078ca259664613bc0e01bc99f4d71f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.107025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a078ca259664613bc0e01bc99f4d71f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.120916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:57.122331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4438740-a0df-41ac-bdb8-b1d367a5030e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a078ca259664613bc0e01bc99f4d71f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.124864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3aaeb3-4676-4291-9c5f-04a750333018 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:57.160159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 910/1800] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:57.163969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.164287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.164569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.182293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 891/1801] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:56 2022] DELETE /identity/v3/users/3a078ca259664613bc0e01bc99f4d71f => generated 0 bytes in 702 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:57.186942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b29144-2b54-4326-b085-98928fbf04b6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.188845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b29144-2b54-4326-b085-98928fbf04b6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.189817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b29144-2b54-4326-b085-98928fbf04b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.251708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-52b29144-2b54-4326-b085-98928fbf04b6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:57.253612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-52b29144-2b54-4326-b085-98928fbf04b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:57.320814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] MFA Rules not processed for user `c6b2dd658b3f4f49ad5ca777b6dff8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:57.322959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:57.407302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 892/1802] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:57.411434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad59a7e-c717-4cf2-943f-0113255c32e8 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.415770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad59a7e-c717-4cf2-943f-0113255c32e8 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.416333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad59a7e-c717-4cf2-943f-0113255c32e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.483545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fad59a7e-c717-4cf2-943f-0113255c32e8 None None] MFA Rules not processed for user `76c0df49fa9342778a56eae098b2f2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:57.487976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fad59a7e-c717-4cf2-943f-0113255c32e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:57.511214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:57.589545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 911/1803] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:57 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:57.592893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:57.603246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:57.662965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91020c73-d293-4f80-a359-eb658ac986c8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:57.677822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:57.678171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.678488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.678765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] PATH_INFO: `/v3/users/00954785920d481495ecfe9f3f0e4767` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.681050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00954785920d481495ecfe9f3f0e4767)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:57.684809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:57.697995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 893/1804] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:57 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:57.701923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.718213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.719039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.807552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:57.813903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:57.837720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00954785920d481495ecfe9f3f0e4767'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.840949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 894/1805] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:57.844605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00954785920d481495ecfe9f3f0e4767'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.845919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00954785920d481495ecfe9f3f0e4767'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.846200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:57.860886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00954785920d481495ecfe9f3f0e4767'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.864506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:57.880832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2946aac0-4d40-4a08-b050-0f7598d6a3e7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:57.893803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:57.894844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.896945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.898054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.899647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:36:57.902742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:57.903917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-07e700aa-b486-40de-81fa-ee3118ca06b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00954785920d481495ecfe9f3f0e4767'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:36:57.905523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:57.944947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 912/1806] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:36:57 2022] DELETE /identity/v3/users/00954785920d481495ecfe9f3f0e4767 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:57.950957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:57.951485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:57.952402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 895/1807] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:36:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:36:57.952995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:57.958868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:57.966148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:58.007422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:58.011050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:58.027254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7afd98-267b-47b7-9b7f-9ad3c4171c90 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:58.219024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 913/1808] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:58.227637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:58.228514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:58.229077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:58.229506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:58.230514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:58.236987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:58.345066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 896/1809] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:57 2022] POST /identity/v3/projects => generated 387 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:58.351346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:58.401417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474be90f-df20-4c1f-bbc1-c518fac3b261 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:58.425070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:58.428242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:58.772901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:58.773663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:58.774643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:58.774928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:58.780220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3f14d9-18bc-481e-a513-9f468fd55425 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:58.781549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:58.786841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:58.787484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:58.787606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:58.787764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:58.788037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:58.832381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:58.839518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:58.889008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:58.906746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 897/1810] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:36:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:36:58.913778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 914/1811] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:36:58 2022] POST /identity/v3/users => generated 294 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:36:58.919138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:58.952192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fa77a1-5e63-497b-a027-ca8910326498 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:58.978145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:58.978639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:58.979021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:58.979433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:58.980157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:58.982026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:59.011591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:59.019724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 898/1812] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:59.029843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cb5feb-561b-47b2-b752-294a558626bb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:59.046105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04847f-38a6-4665-8388-d187db29a636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:59.046105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04847f-38a6-4665-8388-d187db29a636 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:59.046105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04847f-38a6-4665-8388-d187db29a636 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:59.047022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04847f-38a6-4665-8388-d187db29a636 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:59.052820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04847f-38a6-4665-8388-d187db29a636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:59.054743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04847f-38a6-4665-8388-d187db29a636 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:59.105354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 915/1813] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:36:59.109016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:59.125898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4e7886-9946-4884-834d-7f87bf497eb5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:59.142436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e9b0e-7df0-4b56-80c8-c6e92d298500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:59.143128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e9b0e-7df0-4b56-80c8-c6e92d298500 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:59.143434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e9b0e-7df0-4b56-80c8-c6e92d298500 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:59.143762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e9b0e-7df0-4b56-80c8-c6e92d298500 admin admin] PATH_INFO: `/v3/projects/4a6d3415ab3145f69b63287fc93308ff/users/cb892c1fff57473d8bb451bf0c333c02/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:59.189320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e9b0e-7df0-4b56-80c8-c6e92d298500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a6d3415ab3145f69b63287fc93308ff, user_id=cb892c1fff57473d8bb451bf0c333c02, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:59.196336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e9b0e-7df0-4b56-80c8-c6e92d298500 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:36:59.229450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:59.229800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:59.230077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:59.252083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 899/1814] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:36:59 2022] PUT /identity/v3/projects/4a6d3415ab3145f69b63287fc93308ff/users/cb892c1fff57473d8bb451bf0c333c02/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:36:59.269087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:59.270769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:59.274927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:59.410296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] MFA Rules not processed for user `cb892c1fff57473d8bb451bf0c333c02`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:59.412710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:59.486738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:36:59.489546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:59.575993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:36:59.700730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 900/1815] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:36:59.709507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:59.716524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:36:59.783081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 916/1816] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:36:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:36:59.784545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:36:59.950159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17770661-4ea4-4fc8-84f0-08a27e4234bd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:36:59.964914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:36:59.965587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:36:59.965848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:36:59.966116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] PATH_INFO: `/v3/projects/405e982186214adaaf4bc0836213964c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:36:59.968362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=405e982186214adaaf4bc0836213964c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:36:59.970354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:00.014348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994827fb-e0a7-4f02-9cfb-bca84698b1eb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:00.036481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:00.037160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:00.037543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:00.037876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:00.038505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:00.043231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:00.133529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 917/1817] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:36:59 2022] POST /identity/v3/projects => generated 387 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:00.140725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:00.173216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '405e982186214adaaf4bc0836213964c', 'request_id': 'req-bf732803-62b5-40f5-9d19-b27198192a92'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.186455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '405e982186214adaaf4bc0836213964c', 'request_id': 'req-bf732803-62b5-40f5-9d19-b27198192a92'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.206530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '405e982186214adaaf4bc0836213964c', 'request_id': 'req-bf732803-62b5-40f5-9d19-b27198192a92'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.271169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] The token cache is being invalidate because project 405e982186214adaaf4bc0836213964c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:00.271995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.274580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.281070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:00.282544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 901/1818] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:36:59 2022] DELETE /identity/v3/projects/405e982186214adaaf4bc0836213964c => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:00.287584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:00.289696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:00.320406 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4820f30-cf30-4628-8956-b92b9ec17213 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:00.446247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:00.452572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:00.453265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:00.455516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:00.460106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:00.460829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:00.462831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:00.489077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf732803-62b5-40f5-9d19-b27198192a92 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:00.503082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:00.505997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:00.511653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:00.512373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:00.523505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:00.549482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 918/1819] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:00 2022] POST /identity/v3/users => generated 300 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:00.549482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:00.555659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:00.579116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ce2731-daac-4810-802a-3d735913ff19 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:00.585345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:00.586153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:00.586480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:00.586767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] PATH_INFO: `/v3/users/04a34d48279c4348af29b4f65ae00e25` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:00.587994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a34d48279c4348af29b4f65ae00e25)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:00.590374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:00.694469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a34d48279c4348af29b4f65ae00e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.696506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a34d48279c4348af29b4f65ae00e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.699885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a34d48279c4348af29b4f65ae00e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.744367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a34d48279c4348af29b4f65ae00e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.776091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a34d48279c4348af29b4f65ae00e25'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:00.812335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 919/1820] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:00 2022] DELETE /identity/v3/users/04a34d48279c4348af29b4f65ae00e25 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:00.813546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:00.818509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:01.045706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:01.050997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:01.091924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:01.111951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 902/1821] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 829 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:01.158494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ca90c-bca6-4358-8d20-9ffa0b21bf24 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:01.187003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:01.188095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.194666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.194955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:01.195678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:01.197326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:01.231729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 920/1822] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:00 2022] GET /identity/v3/roles => generated 1420 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:01.238081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:01.241095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:01.336698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb74db-ef7b-45e5-bc63-1dd2af2d6c9c service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:01.351273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edde933-8b6d-4390-87ce-0ba9b366bd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:01.353536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde933-8b6d-4390-87ce-0ba9b366bd3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.360436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde933-8b6d-4390-87ce-0ba9b366bd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.361343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edde933-8b6d-4390-87ce-0ba9b366bd3b admin admin] PATH_INFO: `/v3/projects/4b58e99528c64e93bfa9312f2983d933/users/dda5840011124ad28a1c7ccc5f1971e4/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:01.472822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde933-8b6d-4390-87ce-0ba9b366bd3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b58e99528c64e93bfa9312f2983d933, user_id=dda5840011124ad28a1c7ccc5f1971e4, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:01.480598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edde933-8b6d-4390-87ce-0ba9b366bd3b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:01.521869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 903/1823] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:01 2022] PUT /identity/v3/projects/4b58e99528c64e93bfa9312f2983d933/users/dda5840011124ad28a1c7ccc5f1971e4/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:01.524266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:01.566383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d15c94f-7fc6-4b93-93a7-c30d5aaa0951 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:01.750063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81cfdf7-278f-4f0a-a655-b790d2d7424f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:01.752129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfdf7-278f-4f0a-a655-b790d2d7424f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.752460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfdf7-278f-4f0a-a655-b790d2d7424f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.752742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81cfdf7-278f-4f0a-a655-b790d2d7424f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:01.756915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfdf7-278f-4f0a-a655-b790d2d7424f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:01.763984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81cfdf7-278f-4f0a-a655-b790d2d7424f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:01.790700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc126-afad-40c1-85c1-5556f638d2bf None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.791640 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc126-afad-40c1-85c1-5556f638d2bf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.792433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccbc126-afad-40c1-85c1-5556f638d2bf None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:01.795441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 904/1824] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:37:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:37:01.801821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 921/1825] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:01.807948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc126-afad-40c1-85c1-5556f638d2bf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:01.861865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccbc126-afad-40c1-85c1-5556f638d2bf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:01.898161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967edb35-78b9-4fb7-827c-4a284f1e401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:01.899043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967edb35-78b9-4fb7-827c-4a284f1e401b admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.901035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967edb35-78b9-4fb7-827c-4a284f1e401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.901741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967edb35-78b9-4fb7-827c-4a284f1e401b admin admin] PATH_INFO: `/v3/projects/4b58e99528c64e93bfa9312f2983d933/users/dda5840011124ad28a1c7ccc5f1971e4/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:01.913727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967edb35-78b9-4fb7-827c-4a284f1e401b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b58e99528c64e93bfa9312f2983d933, user_id=dda5840011124ad28a1c7ccc5f1971e4, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:01.924358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967edb35-78b9-4fb7-827c-4a284f1e401b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:01.951778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.952068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.954526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:01.974242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 905/1826] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:01 2022] PUT /identity/v3/projects/4b58e99528c64e93bfa9312f2983d933/users/dda5840011124ad28a1c7ccc5f1971e4/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:01.978840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:01.979484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:01.980002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:02.043085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:02.050795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:02.105531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] MFA Rules not processed for user `dda5840011124ad28a1c7ccc5f1971e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:02.108645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:02.322911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 922/1827] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:02.453409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 906/1828] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:01 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:02.476397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:02.477867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:02.790043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162d96da-69cd-4567-bda7-c34e009764a8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:02.825001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:02.835440 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:02.835882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:02.836238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:02.910622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:02.914687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:02.958950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 923/1829] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:02 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:02.972819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:02.974835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:03.032139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9346657-532d-4f21-ae92-1a1904369f7c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:03.067798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:03.068352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db1ff51-e4d3-427e-84a9-9dc8288887d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:03.069407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1ff51-e4d3-427e-84a9-9dc8288887d8 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:03.069910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1ff51-e4d3-427e-84a9-9dc8288887d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:03.070424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db1ff51-e4d3-427e-84a9-9dc8288887d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:03.101532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1ff51-e4d3-427e-84a9-9dc8288887d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:03.103757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db1ff51-e4d3-427e-84a9-9dc8288887d8 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:03.143774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 907/1830] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:03.174991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:03.314439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f064c706-4eea-42fe-b113-9851aa102a1e service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:03.343452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:03.343995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:03.344291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:03.346476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] PATH_INFO: `/v3/users/cb892c1fff57473d8bb451bf0c333c02` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:03.349587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb892c1fff57473d8bb451bf0c333c02)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:03.355322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:03.527055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb892c1fff57473d8bb451bf0c333c02'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:03.527670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb892c1fff57473d8bb451bf0c333c02'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:03.528341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb892c1fff57473d8bb451bf0c333c02'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:03.555247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb892c1fff57473d8bb451bf0c333c02'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:03.621581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb892c1fff57473d8bb451bf0c333c02'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:03.675870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 924/1831] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:03 2022] DELETE /identity/v3/users/cb892c1fff57473d8bb451bf0c333c02 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:03.681290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:03.681880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:03.682498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:03.740997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:03.746632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:03.749042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:03.764200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:03.823237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:04.016976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e575b-f5e6-4196-a296-aa2bfebebd01 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:04.027322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 908/1832] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:04.043965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:04.053235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:04.067379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:04.070447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:04.072932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:04.080963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] PATH_INFO: `/v3/projects/c2cac24d02c641be9e27258960f6b204` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:04.090875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2cac24d02c641be9e27258960f6b204)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:04.094681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:04.112217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b563a766-1713-4283-ae5d-82a7611820fa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:04.142709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:04.144111 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:04.151939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:04.152277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] PATH_INFO: `/v3/projects/1f8d6d7b96264e1fbac7ac9bbf30c733` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:04.158947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f8d6d7b96264e1fbac7ac9bbf30c733)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:04.170123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:04.316667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:04.320233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2cac24d02c641be9e27258960f6b204', 'request_id': 'req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.339536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2cac24d02c641be9e27258960f6b204', 'request_id': 'req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.357693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2cac24d02c641be9e27258960f6b204', 'request_id': 'req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.385435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] The token cache is being invalidate because project c2cac24d02c641be9e27258960f6b204 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:04.388742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.390770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.396812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 925/1833] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:03 2022] DELETE /identity/v3/projects/c2cac24d02c641be9e27258960f6b204 => generated 0 bytes in 648 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:04.398302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:04.400154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:04.407586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f8d6d7b96264e1fbac7ac9bbf30c733', 'request_id': 'req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.439121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f8d6d7b96264e1fbac7ac9bbf30c733', 'request_id': 'req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.471677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f8d6d7b96264e1fbac7ac9bbf30c733', 'request_id': 'req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.528926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] The token cache is being invalidate because project 1f8d6d7b96264e1fbac7ac9bbf30c733 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:04.530494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.533639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:04.537568 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 909/1834] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:04 2022] DELETE /identity/v3/projects/1f8d6d7b96264e1fbac7ac9bbf30c733 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:04.541792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:04.552795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:04.572811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c00bbcb-30a3-42a2-bc4d-e4c898b2769b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:04.861510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ae7ef-1b35-4798-8371-72c4aee5fc50 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:04.869794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:04.870693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:04.870973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:04.871604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] PATH_INFO: `/v3/users/55884db659f84f12bc28238d85d1c818` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:04.873477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55884db659f84f12bc28238d85d1c818)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:04.877167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:04.912468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:04.913100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:04.913412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:04.913644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:04.916039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:04.967961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:04.970612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbde191-ff8b-419a-be86-21b7eb87a68d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:05.011907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 926/1835] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 614 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:05.013532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48916-9c4b-41c7-8dd2-b9de737ff6be None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:05.013797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48916-9c4b-41c7-8dd2-b9de737ff6be None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:05.014093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de48916-9c4b-41c7-8dd2-b9de737ff6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:05.046934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:05.053832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55884db659f84f12bc28238d85d1c818'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:05.056537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55884db659f84f12bc28238d85d1c818'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:05.060459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55884db659f84f12bc28238d85d1c818'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:05.080597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4de48916-9c4b-41c7-8dd2-b9de737ff6be None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:05.082499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4de48916-9c4b-41c7-8dd2-b9de737ff6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:05.100784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55884db659f84f12bc28238d85d1c818'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:05.125364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55884db659f84f12bc28238d85d1c818'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:05.146868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 927/1836] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:05.159451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:05.160082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:05.161040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:05.187270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:05.189092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:05.201536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 910/1837] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:04 2022] DELETE /identity/v3/users/55884db659f84f12bc28238d85d1c818 => generated 0 bytes in 661 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:05.211908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 928/1838] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:05.212322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:05.414600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493095d-544a-434e-bc9f-1fa2c4478076 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:05.430203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:05.431342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:05.432679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:05.434038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:05.436479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:05.439128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:05.444621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:05.503631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:05.509353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 911/1839] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:05.523397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b93419-3063-4a97-9016-91239a791dc1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:05.536959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5250fc9c-0d99-44fc-9a06-a0e57e0eae51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:05.537690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250fc9c-0d99-44fc-9a06-a0e57e0eae51 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:05.541652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250fc9c-0d99-44fc-9a06-a0e57e0eae51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:05.542017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5250fc9c-0d99-44fc-9a06-a0e57e0eae51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:05.542915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250fc9c-0d99-44fc-9a06-a0e57e0eae51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:05.550053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5250fc9c-0d99-44fc-9a06-a0e57e0eae51 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:05.644615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:05.652818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 929/1840] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:05 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:05.665280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb50e91-3182-474e-a5c9-b8c4274650e1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:05.763170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:05.763909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:05.764408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:05.837542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] MFA Rules not processed for user `5ae7930c17434ae1b6df030b0599e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:05.844758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:05.963293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:05.964430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:05.964904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:05.965204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:05.965853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:05.967911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:05.984291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 930/1841] 10.222.0.75 () {58 vars in 1015 bytes} [Wed Aug 24 03:37:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:06.012040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 912/1842] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:05 2022] POST /identity/v3/users => generated 288 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:06.018005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:06.028869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:06.031085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:06.073852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20b9a32-24f7-43e5-8b8c-d16819b36793 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:06.081501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:06.083438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:06.084254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:06.084434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:06.086699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e7cc9-b1aa-4d43-a1ca-8fd8526ac18b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:06.113516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:06.125468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:06.126591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:06.126761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:06.126985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:06.127460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:06.128662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:06.134228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:06.138478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:06.180727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 913/1843] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:06 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:06.187648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 931/1844] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:06.192756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:06.223494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8e3bb5-fa7e-45a7-af1a-1f617a377b94 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:06.236548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cbd579-17cd-424d-a81e-9e1aee4814b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:06.237727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbd579-17cd-424d-a81e-9e1aee4814b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:06.243061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbd579-17cd-424d-a81e-9e1aee4814b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:06.243306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cbd579-17cd-424d-a81e-9e1aee4814b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:06.244938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbd579-17cd-424d-a81e-9e1aee4814b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:06.248887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cbd579-17cd-424d-a81e-9e1aee4814b5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:06.295632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 914/1845] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:06 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:06.299794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:06.326851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798b79ee-31e0-43b2-80fb-5c43aa976a5e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:06.353371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:06.355600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:06.355928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:06.357557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] PATH_INFO: `/v3/projects/6d147de1407f425e91ee57a4be743179/users/7f13c3c09c2b48d58924139701a85084/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:06.388378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d147de1407f425e91ee57a4be743179, user_id=7f13c3c09c2b48d58924139701a85084, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:06.396398 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:06.439768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 932/1846] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:06 2022] PUT /identity/v3/projects/6d147de1407f425e91ee57a4be743179/users/7f13c3c09c2b48d58924139701a85084/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:06.443233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:06.443799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:06.444136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:06.546997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] MFA Rules not processed for user `7f13c3c09c2b48d58924139701a85084`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:06.549157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:06.583348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:06.813357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 915/1847] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:06.831281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:06.840163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:07.153120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d062c8-aa78-4a47-a7f5-d672630d1545 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:07.163148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b64ec09-7f32-4aec-a660-368c7f2b45fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:07.165001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64ec09-7f32-4aec-a660-368c7f2b45fb service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:07.165647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64ec09-7f32-4aec-a660-368c7f2b45fb service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:07.166367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b64ec09-7f32-4aec-a660-368c7f2b45fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:07.188184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64ec09-7f32-4aec-a660-368c7f2b45fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:07.191519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b64ec09-7f32-4aec-a660-368c7f2b45fb service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:07.230324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 933/1848] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:06 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:09.080943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:09.084280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:09.262658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3b8ee-e765-443c-abda-b48bf2ca2a16 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:09.279070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:09.280415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:09.280978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:09.281613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] PATH_INFO: `/v3/users/25bee7c69d074709a2ffa3ffbe7cd864` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:09.284266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25bee7c69d074709a2ffa3ffbe7cd864)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:09.289969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:09.399297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:09.399998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:09.400814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:09.452883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25bee7c69d074709a2ffa3ffbe7cd864'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:09.454436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25bee7c69d074709a2ffa3ffbe7cd864'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:09.455971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25bee7c69d074709a2ffa3ffbe7cd864'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:09.456641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:09.460117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:09.471565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25bee7c69d074709a2ffa3ffbe7cd864'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:09.493502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25bee7c69d074709a2ffa3ffbe7cd864'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:09.540333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 916/1849] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:09 2022] DELETE /identity/v3/users/25bee7c69d074709a2ffa3ffbe7cd864 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:09.604203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 934/1850] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:37:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:09.628617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:09.631982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:09.836959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:09.963362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de733a0b-3aae-4007-a720-35a3da310123 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:09.997845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:10.000329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:10.002112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:10.002683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:10.287973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:10.299558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:10.334046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 917/1851] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:37:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:37:10.607055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:10.608882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:10.634087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:10.636435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:10.679668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24553df3-8e89-4069-b207-2335bf63f1f5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:10.868467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:10.870004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:10.871506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:10.875634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] PATH_INFO: `/v3/projects/4a6d3415ab3145f69b63287fc93308ff` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:10.880567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a6d3415ab3145f69b63287fc93308ff)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:10.899427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:10.902388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78931e60-1bc2-45ff-9561-18ae0bf2c329 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:10.907443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:10.915963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:10.916686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:10.917018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:10.917305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] PATH_INFO: `/v3/projects/bcc0b46c5ff543d884811206546d4fc7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:10.928360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc0b46c5ff543d884811206546d4fc7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:10.931288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:11.120553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a6d3415ab3145f69b63287fc93308ff', 'request_id': 'req-93fedf62-7fdc-4760-bd27-599bc50be7d2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.150646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a6d3415ab3145f69b63287fc93308ff', 'request_id': 'req-93fedf62-7fdc-4760-bd27-599bc50be7d2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.164851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc0b46c5ff543d884811206546d4fc7', 'request_id': 'req-48d13ba5-06be-435f-9a18-112a0045422f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.200630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a6d3415ab3145f69b63287fc93308ff', 'request_id': 'req-93fedf62-7fdc-4760-bd27-599bc50be7d2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.202389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc0b46c5ff543d884811206546d4fc7', 'request_id': 'req-48d13ba5-06be-435f-9a18-112a0045422f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.230612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc0b46c5ff543d884811206546d4fc7', 'request_id': 'req-48d13ba5-06be-435f-9a18-112a0045422f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.247171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] The token cache is being invalidate because project 4a6d3415ab3145f69b63287fc93308ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:11.248866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.251545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.261943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 935/1852] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:10 2022] DELETE /identity/v3/projects/4a6d3415ab3145f69b63287fc93308ff => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:11.267038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:11.276034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:11.278588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] The token cache is being invalidate because project bcc0b46c5ff543d884811206546d4fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:11.290727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.298757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-48d13ba5-06be-435f-9a18-112a0045422f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.304274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 918/1853] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:10 2022] DELETE /identity/v3/projects/bcc0b46c5ff543d884811206546d4fc7 => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:11.305884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e638d-ed3c-46f9-85a5-2152132e7532 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.306164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e638d-ed3c-46f9-85a5-2152132e7532 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.306546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e638d-ed3c-46f9-85a5-2152132e7532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.335523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:11.371680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-da6e638d-ed3c-46f9-85a5-2152132e7532 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:11.373373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-da6e638d-ed3c-46f9-85a5-2152132e7532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:11.465575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fedf62-7fdc-4760-bd27-599bc50be7d2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:11.558137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 919/1854] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:11.568845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697903-8303-446b-8900-b39c768f8dbf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.569128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697903-8303-446b-8900-b39c768f8dbf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.569582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95697903-8303-446b-8900-b39c768f8dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.633188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:11.635431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.636486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.637210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] PATH_INFO: `/v3/users/dda5840011124ad28a1c7ccc5f1971e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.640588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dda5840011124ad28a1c7ccc5f1971e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:11.645253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:11.659528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-95697903-8303-446b-8900-b39c768f8dbf None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:11.662003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-95697903-8303-446b-8900-b39c768f8dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:11.724280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 920/1855] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:11.730257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95697903-8303-446b-8900-b39c768f8dbf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:11.762545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95697903-8303-446b-8900-b39c768f8dbf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:11.785076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:11.803225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dda5840011124ad28a1c7ccc5f1971e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.803225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dda5840011124ad28a1c7ccc5f1971e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.803809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dda5840011124ad28a1c7ccc5f1971e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.820680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:11.821324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:11.821613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:11.821889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:11.822622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:11.822929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:11.824661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:11.838349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dda5840011124ad28a1c7ccc5f1971e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.863774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dda5840011124ad28a1c7ccc5f1971e4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:11.890838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 936/1856] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:11 2022] DELETE /identity/v3/users/dda5840011124ad28a1c7ccc5f1971e4 => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:11.894487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 921/1857] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:11.908243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:11.911609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:12.119089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aa8b72-c6cc-44e7-bf6e-f2700cfb9bcc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:12.132466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:12.132466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:12.132466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:12.133322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:12.133322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:12.138433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:12.245613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 937/1858] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:11 2022] POST /identity/v3/projects => generated 389 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:12.249346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:12.263995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642bf510-6447-4af4-b241-5783821b12f5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:12.498976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:12.500434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:12.500898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:12.501332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:12.502238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:12.509401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:12.599217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 922/1859] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:12 2022] POST /identity/v3/users => generated 295 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:12.601281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:12.619363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d562626-66ec-4cb1-a207-3611bebbce5c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:12.637966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:12.639434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:12.640034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:12.640500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:12.641390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:12.643594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:12.701078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:12.701837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 938/1860] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:12.724070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1566d551-9d61-4302-827e-eae1f7f67d11 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:12.741952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daef09e9-87bc-4df3-927f-f01cf44073df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:12.742810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef09e9-87bc-4df3-927f-f01cf44073df admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:12.743202 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef09e9-87bc-4df3-927f-f01cf44073df admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:12.743666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daef09e9-87bc-4df3-927f-f01cf44073df admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:12.744549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef09e9-87bc-4df3-927f-f01cf44073df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:12.746449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daef09e9-87bc-4df3-927f-f01cf44073df admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:12.776379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 923/1861] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:12 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:12.777962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:12.813940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aefa6b-cec1-4862-a6f0-704633acadf1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:12.836334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:12.839906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:12.842775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:12.843356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] PATH_INFO: `/v3/projects/755d5b6317d54cdbaa87a1a306d55891/users/cfcfdca81d3b4bb39b8bfd6b2019844f/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:12.916961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=755d5b6317d54cdbaa87a1a306d55891, user_id=cfcfdca81d3b4bb39b8bfd6b2019844f, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:12.924044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:12.974238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 939/1862] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:12 2022] PUT /identity/v3/projects/755d5b6317d54cdbaa87a1a306d55891/users/cfcfdca81d3b4bb39b8bfd6b2019844f/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:12.985229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:12.985607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:12.986014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:13.086970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] MFA Rules not processed for user `cfcfdca81d3b4bb39b8bfd6b2019844f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:13.088244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:13.254570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:13.299655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 924/1863] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:13.301413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:13.384423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:13.393825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:13.772071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c69d86-fc90-4a89-86eb-e93c4d48e884 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:13.816303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:13.819785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:13.829829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:13.830491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:13.831722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:13.853556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:13.928524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:13.931120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0fab63-d96f-4098-8045-a7338398bc67 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:13.954993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:13.968197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:13.969397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:13.970312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] PATH_INFO: `/v3/projects/d270caa6372a41c59350c486d2de18c8` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:13.973416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d270caa6372a41c59350c486d2de18c8)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:13.975773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 940/1864] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:13 2022] POST /identity/v3/projects => generated 391 bytes in 675 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:13.977241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:13.978694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.189956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd270caa6372a41c59350c486d2de18c8', 'request_id': 'req-469fbf05-13ae-4256-ae77-52d6f81ea123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.194627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41bd0-4c93-4314-b453-7e1a0f2b8d5f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.204779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd270caa6372a41c59350c486d2de18c8', 'request_id': 'req-469fbf05-13ae-4256-ae77-52d6f81ea123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.222472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd270caa6372a41c59350c486d2de18c8', 'request_id': 'req-469fbf05-13ae-4256-ae77-52d6f81ea123'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.239718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] The token cache is being invalidate because project d270caa6372a41c59350c486d2de18c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:14.253735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.257899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.266634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 925/1865] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:13 2022] DELETE /identity/v3/projects/d270caa6372a41c59350c486d2de18c8 => generated 0 bytes in 881 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:14.267912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.270751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.554865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4e460a-dff5-463c-b5c8-ee26ab75dc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.555853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e460a-dff5-463c-b5c8-ee26ab75dc4c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.556323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e460a-dff5-463c-b5c8-ee26ab75dc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.556797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4e460a-dff5-463c-b5c8-ee26ab75dc4c admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.557913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e460a-dff5-463c-b5c8-ee26ab75dc4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.561570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4e460a-dff5-463c-b5c8-ee26ab75dc4c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.586569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fbf05-13ae-4256-ae77-52d6f81ea123 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.643699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.649717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.650904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.651684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] PATH_INFO: `/v3/users/c6b2dd658b3f4f49ad5ca777b6dff8b1` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.653475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 941/1866] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:13 2022] POST /identity/v3/users => generated 302 bytes in 677 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:14.656144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6b2dd658b3f4f49ad5ca777b6dff8b1)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.656614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e45bea-a1b7-45d9-babd-7ca9f8e05e95 None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.657155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e45bea-a1b7-45d9-babd-7ca9f8e05e95 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.657635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e45bea-a1b7-45d9-babd-7ca9f8e05e95 None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.661075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 942/1867] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:37:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:37:14.662415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e45bea-a1b7-45d9-babd-7ca9f8e05e95 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:14.665152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-24e45bea-a1b7-45d9-babd-7ca9f8e05e95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:14.680647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.770465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e45bea-a1b7-45d9-babd-7ca9f8e05e95 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:14.791133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9cd76-ca03-4d33-9255-6765b4f84ef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:14.793267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9cd76-ca03-4d33-9255-6765b4f84ef4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.794512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9cd76-ca03-4d33-9255-6765b4f84ef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.794878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9cd76-ca03-4d33-9255-6765b4f84ef4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.800270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9cd76-ca03-4d33-9255-6765b4f84ef4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:14.823659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9cd76-ca03-4d33-9255-6765b4f84ef4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:14.839717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:14.927468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6b2dd658b3f4f49ad5ca777b6dff8b1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.929274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6b2dd658b3f4f49ad5ca777b6dff8b1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.935089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6b2dd658b3f4f49ad5ca777b6dff8b1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:14.957868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 943/1868] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:14 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:14.959438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:14.959738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:14.962552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:14.966312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6b2dd658b3f4f49ad5ca777b6dff8b1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:15.000856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6b2dd658b3f4f49ad5ca777b6dff8b1'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:15.086927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 926/1869] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:14 2022] DELETE /identity/v3/users/c6b2dd658b3f4f49ad5ca777b6dff8b1 => generated 0 bytes in 819 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:15.088021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:15.090000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:15.126784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:15.134603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:15.353348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c81e8f-c212-44c3-92dd-b3767bb7625e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:15.371218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:15.372149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.372598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.373911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] PATH_INFO: `/v3/projects/199422ae7e7442f189207ccedf5be394/users/4a88e21c9db249b7a5b3b09c6bdf9a1b/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:15.427944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 944/1870] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:37:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:15.435674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=199422ae7e7442f189207ccedf5be394, user_id=4a88e21c9db249b7a5b3b09c6bdf9a1b, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:15.443135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:15.488853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 927/1871] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:15 2022] PUT /identity/v3/projects/199422ae7e7442f189207ccedf5be394/users/4a88e21c9db249b7a5b3b09c6bdf9a1b/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:15.496963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:15.522600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fb0717-aa9c-465e-ad68-bc783a9cce8a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:15.534124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:15.714165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e16548c-5be0-430b-92a8-ec7ee5773f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:15.715859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16548c-5be0-430b-92a8-ec7ee5773f70 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.717027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16548c-5be0-430b-92a8-ec7ee5773f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.717898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e16548c-5be0-430b-92a8-ec7ee5773f70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:15.720816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16548c-5be0-430b-92a8-ec7ee5773f70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:15.727714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e16548c-5be0-430b-92a8-ec7ee5773f70 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:15.772139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 945/1872] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:15 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:15.772308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:15.792052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b137772d-1c8f-4bb5-a563-08721dc3ce01 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:15.808759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:15.810361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.810934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.811930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] PATH_INFO: `/v3/projects/199422ae7e7442f189207ccedf5be394/users/4a88e21c9db249b7a5b3b09c6bdf9a1b/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:15.815181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=199422ae7e7442f189207ccedf5be394, user_id=4a88e21c9db249b7a5b3b09c6bdf9a1b, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:15.822552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:15.896504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 928/1873] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:15 2022] PUT /identity/v3/projects/199422ae7e7442f189207ccedf5be394/users/4a88e21c9db249b7a5b3b09c6bdf9a1b/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:15.917597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:15.919041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:15.919546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:16.053653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] MFA Rules not processed for user `4a88e21c9db249b7a5b3b09c6bdf9a1b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:16.055132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:16.328891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 946/1874] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 413 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:16.357555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:16.360188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:16.549281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee781e-7e69-491d-8961-0bd16ec906bc admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:16.564355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f7bd9-56ea-49cb-a945-45092fe2e5aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:16.567272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f7bd9-56ea-49cb-a945-45092fe2e5aa service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:16.567819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f7bd9-56ea-49cb-a945-45092fe2e5aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:16.571403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f7bd9-56ea-49cb-a945-45092fe2e5aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:16.618284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f7bd9-56ea-49cb-a945-45092fe2e5aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:16.623534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f7bd9-56ea-49cb-a945-45092fe2e5aa service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:16.655072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 929/1875] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:16 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:16.764556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:16.766693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:16.790960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7997fc-c7c2-4ee2-abd9-dcf028753c5b None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:16.791704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7997fc-c7c2-4ee2-abd9-dcf028753c5b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:16.792348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7997fc-c7c2-4ee2-abd9-dcf028753c5b None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:16.806417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 930/1876] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:37:16 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 24 03:37:16.826557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd334d-43c1-4f21-9ce2-54641b5ff1bc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:16.833963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:16.836253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:16.837706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:16.838251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:16.839225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:16.840394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:16.841267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:16.847226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:16.912028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:16.914902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:16.934210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:17.056682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:17.059791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:17.089958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 947/1877] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:17.092694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:17.109120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 931/1878] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:37:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:17.116225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:17.177592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7d011d-cbd1-4645-9f24-d0f84e23d277 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:17.185703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:17.186378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:17.186661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:17.186943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:17.199050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:17.201056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:17.214673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 948/1879] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:17.419546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:17.438274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c379f-2103-4c03-bf09-459d01e6019a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:17.507326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:17.508158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:17.508530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:17.508870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:17.521477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:17.524180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:17.541581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 932/1880] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:17.625332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:17.625722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:17.683775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:17.788164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4078-b9db-446e-8fa7-7418420ad337 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:17.804149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:17.805308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:17.805803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:17.806303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] PATH_INFO: `/v3/projects/4b58e99528c64e93bfa9312f2983d933` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:17.808746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b58e99528c64e93bfa9312f2983d933)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:17.814123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:17.986546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:17.989029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:18.005479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b58e99528c64e93bfa9312f2983d933', 'request_id': 'req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.026745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b58e99528c64e93bfa9312f2983d933', 'request_id': 'req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.031076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848c18fa-8afd-40b6-b89b-fa4a75385a7d service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.051297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b58e99528c64e93bfa9312f2983d933', 'request_id': 'req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.084985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] The token cache is being invalidate because project 4b58e99528c64e93bfa9312f2983d933 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:18.084985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.086327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da69b8b7-61aa-4c96-ac7f-f2d4e7a900a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.087312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 949/1881] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:17 2022] DELETE /identity/v3/projects/4b58e99528c64e93bfa9312f2983d933 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:18.093813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feebbb3-e3b2-4332-b909-92c45a54b44b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.094113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feebbb3-e3b2-4332-b909-92c45a54b44b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.094527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feebbb3-e3b2-4332-b909-92c45a54b44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.124537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3feebbb3-e3b2-4332-b909-92c45a54b44b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:18.126967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3feebbb3-e3b2-4332-b909-92c45a54b44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:18.203377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.204863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.214082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.215604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] PATH_INFO: `/v3/projects/5374bd1ac87b41dcb313cac4e190ca88` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.219227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] RBAC: Authorizing `identity:delete_project(project_id=5374bd1ac87b41dcb313cac4e190ca88)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.224511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.341647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5374bd1ac87b41dcb313cac4e190ca88', 'request_id': 'req-6a4e28f9-448f-4a7e-a814-30cfb8170cec'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.352998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5374bd1ac87b41dcb313cac4e190ca88', 'request_id': 'req-6a4e28f9-448f-4a7e-a814-30cfb8170cec'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.354366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 950/1882] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:18.355886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.356229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.356444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.374317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5374bd1ac87b41dcb313cac4e190ca88', 'request_id': 'req-6a4e28f9-448f-4a7e-a814-30cfb8170cec'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.385695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:18.389782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:18.397886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] The token cache is being invalidate because project 5374bd1ac87b41dcb313cac4e190ca88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:18.398613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.399721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6a4e28f9-448f-4a7e-a814-30cfb8170cec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:18.402417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 933/1883] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:17 2022] DELETE /identity/v3/projects/5374bd1ac87b41dcb313cac4e190ca88 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:18.409272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da008cfc-0594-45e9-b620-e6c0f8302402 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.409634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da008cfc-0594-45e9-b620-e6c0f8302402 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.409989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da008cfc-0594-45e9-b620-e6c0f8302402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.468342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-da008cfc-0594-45e9-b620-e6c0f8302402 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:18.470704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-da008cfc-0594-45e9-b620-e6c0f8302402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:18.488312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:18.519021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:18.532183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 951/1884] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:18.538688 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:18.554935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887624fd-fcfd-4ea6-83ff-c6089511f669 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.560682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.561372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.561661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.561933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.562646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:18.562952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.564602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.580169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 952/1885] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:18.586558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:18.610574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679c148c-0641-4b25-86f0-a009d9d2644b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.619125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.620415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.621004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.622208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.623200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.627317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.634392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 934/1886] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:18.639753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.642946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.645762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.714656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:18.714656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:18.729959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 953/1887] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/projects => generated 401 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:18.736621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 935/1888] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:18.737467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:18.742513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:18.757030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785427bf-fb57-4ff0-bcfb-c2cf1f9a7678 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.878143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977bbf8a-e6e2-4c41-aeb5-b63ffa7d452f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.887073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.887629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.888026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.888557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.888726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.889054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.889219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.889695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.890175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:18.890927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.891242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.893782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.895485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.921459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 936/1889] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:18.929311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:18.936059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 954/1890] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/users => generated 301 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:18.937492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:18.945593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3ce0f0-b72f-425a-9054-b9e1f018cdfe admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.955303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.955625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084efc23-60c3-4b42-95d6-894696707514 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:18.956311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.956754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.957961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.959407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.963053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.965349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:18.966489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:18.966940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:18.967376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:18.968520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:18.975899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:18.996946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 955/1891] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:18 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:19.001619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.012746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca4b6b8-5019-4b55-ad37-8b81307a20be admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.023759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 937/1892] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:18 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:19.024040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.024040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.024040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.024040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.024622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.024622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.027201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.037355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 956/1893] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:19.042018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.053157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84035151-60c9-45e9-8a88-68cab76390dc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.131484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cee2324-c1f4-4cfa-8e54-27f5a860e47a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.140551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.141426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.141820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.142270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.143241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.147216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.154966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efe1b1-e4cc-4355-b7bb-fea48d492af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.155980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efe1b1-e4cc-4355-b7bb-fea48d492af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.156557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efe1b1-e4cc-4355-b7bb-fea48d492af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.157014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efe1b1-e4cc-4355-b7bb-fea48d492af5 admin admin] PATH_INFO: `/v3/projects/cfd1a32424914d6a90e6f9e3719dfeb3/users/fb7c120737664d55b4fb9d2ac1492291/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.179699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efe1b1-e4cc-4355-b7bb-fea48d492af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfd1a32424914d6a90e6f9e3719dfeb3, user_id=fb7c120737664d55b4fb9d2ac1492291, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.182591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efe1b1-e4cc-4355-b7bb-fea48d492af5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.182783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 938/1894] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/users => generated 291 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:19.188750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.200587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 957/1895] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:19 2022] PUT /identity/v3/projects/cfd1a32424914d6a90e6f9e3719dfeb3/users/fb7c120737664d55b4fb9d2ac1492291/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:19.209511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bac2d5-2c56-4b4b-beb7-194995ec973a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.216191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.216647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.217118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.231927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:19.239690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:19.282515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] MFA Rules not processed for user `fb7c120737664d55b4fb9d2ac1492291`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:19.284420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:19.347066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.348934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.349308 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.350030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.351408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.353347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.376861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 939/1896] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:19.378377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.394071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc3544-24a9-439d-80e6-ba268cef72bf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.403337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.404366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.404836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.405616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.406790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.410647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.440273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 940/1897] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:19.441587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.449870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 958/1898] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:19.458554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.473750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e85c553-ebdb-4ab3-bc35-337a89c215a0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.481206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb695d0-47c9-438e-858f-0955f60b73a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.482089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb695d0-47c9-438e-858f-0955f60b73a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.482528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb695d0-47c9-438e-858f-0955f60b73a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.482942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb695d0-47c9-438e-858f-0955f60b73a1 admin admin] PATH_INFO: `/v3/projects/a2f0d79da8744ed9a741c91025c87b48/users/41b2ea939e8043fbb7ae10d2bf085476/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.487468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef2790-e25f-4dbc-9661-0634fa50a09e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.494872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.495859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.496396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.497160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.498307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.503277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.528133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb695d0-47c9-438e-858f-0955f60b73a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f0d79da8744ed9a741c91025c87b48, user_id=41b2ea939e8043fbb7ae10d2bf085476, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.533848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb695d0-47c9-438e-858f-0955f60b73a1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.559044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 941/1899] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:19 2022] PUT /identity/v3/projects/a2f0d79da8744ed9a741c91025c87b48/users/41b2ea939e8043fbb7ae10d2bf085476/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:19.576958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 959/1900] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:19.577218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.577218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.577218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.578482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.629635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] MFA Rules not processed for user `41b2ea939e8043fbb7ae10d2bf085476`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:19.630970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:19.709641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfeb43e-35a5-40ef-8b92-e0b83d716e83 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.730622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.735351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.736057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.736807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.739343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.743049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.760353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 942/1901] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:19.765192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.792656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 960/1902] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/users => generated 306 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:19.794904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44060bd4-ed8c-401d-a797-a43c2086cb4b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.799195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.803492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.804294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.804717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.805149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.805921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.808023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.817011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aed968-ddd0-499c-8b92-f0cfaede34b6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.823342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.824681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.824964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.825240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.826260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:19.827914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:19.859992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 943/1903] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:19.860915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.870111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 961/1904] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:19 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:19.873476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:19.893217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb18a5b-8410-4ead-a480-e7b8fbeb43df admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.936563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:19.973828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174a7602-f9c5-42ce-81d1-50be44900014 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:19.989976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:19.993124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:19.994049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:19.994508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:19.994914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:19.996009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.000984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.021149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.027395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.027797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.028262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] PATH_INFO: `/v3/projects/ff65f6271f5142debc630b8ff631492e/users/50df9c8cbc3c4de9b533cc5b1e4f2168/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.071371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff65f6271f5142debc630b8ff631492e, user_id=50df9c8cbc3c4de9b533cc5b1e4f2168, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.074153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.074522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 944/1905] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:19 2022] POST /identity/v3/users => generated 296 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:20.076178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.107539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3524f648-fce8-430b-bcfc-d60fb8da3654 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.117749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.119039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.119731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.120177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.121146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.123112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 962/1906] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:19 2022] PUT /identity/v3/projects/ff65f6271f5142debc630b8ff631492e/users/50df9c8cbc3c4de9b533cc5b1e4f2168/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:20.123307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.130040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.142803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 945/1907] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:20.148217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.166952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c7c0a7-3a17-40a5-a6ea-ba80a093bbe4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.262040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35af4b4-2dd0-4d11-be0d-8bb719b9fc59 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.270167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.270875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.271580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.271580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] PATH_INFO: `/v3/projects/b26e1b5455bd47c1b488604a66feb697/users/7ed9ddb9e7e34a8bbfc404237345c416/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.272790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.273680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.273680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.273991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.275150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.278248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.297975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26e1b5455bd47c1b488604a66feb697, user_id=7ed9ddb9e7e34a8bbfc404237345c416, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.301853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.302380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 963/1908] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:20.306920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.324126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49f8f35-8bf9-431c-b00c-44c20c7184e5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.328833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 946/1909] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:20 2022] PUT /identity/v3/projects/b26e1b5455bd47c1b488604a66feb697/users/7ed9ddb9e7e34a8bbfc404237345c416/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:20.337705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.433519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fda116-4e1c-447d-8cc9-ccb0267fbd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.433519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fda116-4e1c-447d-8cc9-ccb0267fbd82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.433519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fda116-4e1c-447d-8cc9-ccb0267fbd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.433519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fda116-4e1c-447d-8cc9-ccb0267fbd82 admin admin] PATH_INFO: `/v3/projects/ff65f6271f5142debc630b8ff631492e/users/50df9c8cbc3c4de9b533cc5b1e4f2168/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.436517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fda116-4e1c-447d-8cc9-ccb0267fbd82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff65f6271f5142debc630b8ff631492e, user_id=50df9c8cbc3c4de9b533cc5b1e4f2168, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.441188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fda116-4e1c-447d-8cc9-ccb0267fbd82 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.468463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 964/1910] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:20 2022] PUT /identity/v3/projects/ff65f6271f5142debc630b8ff631492e/users/50df9c8cbc3c4de9b533cc5b1e4f2168/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:20.485427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.485860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.486249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.503399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5159b1a-1b1b-4e6c-bc5b-973c2d2d548a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.532258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] MFA Rules not processed for user `50df9c8cbc3c4de9b533cc5b1e4f2168`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:20.534176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:20.597382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:20.623336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.624272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.624725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.625208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.626100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.627869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.646938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 947/1911] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:20 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:20.654133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:20.654461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.685217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa2a8b2-dd86-404e-9542-1f6e3655bacf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.691291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4feadaf8-9ab1-4ace-9e50-7e33a4716315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.692096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feadaf8-9ab1-4ace-9e50-7e33a4716315 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.692520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feadaf8-9ab1-4ace-9e50-7e33a4716315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.692937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4feadaf8-9ab1-4ace-9e50-7e33a4716315 admin admin] PATH_INFO: `/v3/projects/b26e1b5455bd47c1b488604a66feb697/users/7ed9ddb9e7e34a8bbfc404237345c416/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.695996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feadaf8-9ab1-4ace-9e50-7e33a4716315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b26e1b5455bd47c1b488604a66feb697, user_id=7ed9ddb9e7e34a8bbfc404237345c416, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:20.701524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4feadaf8-9ab1-4ace-9e50-7e33a4716315 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:20.711949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 965/1912] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:20 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:20.729298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:20.731759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:20.741314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 948/1913] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:20 2022] PUT /identity/v3/projects/b26e1b5455bd47c1b488604a66feb697/users/7ed9ddb9e7e34a8bbfc404237345c416/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:20.757492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.758589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.759181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:20.824353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1f0d3-fc56-42f1-9949-463b23193ad9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:20.866963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] MFA Rules not processed for user `7ed9ddb9e7e34a8bbfc404237345c416`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:20.868337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:20.895152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:20.896807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:20.898392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:20.899458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:21.025630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:21.029645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:21.050771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 949/1914] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:20 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:21.052103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 966/1915] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:20 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:21.104520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:21.117039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb046c-ed17-40e2-9829-d79d64f124d9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:21.122616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:21.124129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:21.124521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:21.125151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:21.157263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:21.162331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:21.188794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 950/1916] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:21 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:21.309735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:21.331523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098feea-4a5d-4ab4-b620-b7767c30a58f service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:21.342874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:21.344469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:21.345189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:21.345902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:21.458681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:21.479049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:21.481995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:21.504581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 967/1917] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:21 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:22.101449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:22.121550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e801132f-0712-421a-b763-f23565ca5d32 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:22.130065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed9ddb9e7e34a8bbfc404237345c416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26e1b5455bd47c1b488604a66feb697', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:22.131307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:22.131807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:22.132295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:22.133515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:22.134073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:22.136903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:22.170967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 951/1918] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:22.177715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:22.194810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb4c440-7616-436f-9d8a-abfe20fff413 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:22.209777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed9ddb9e7e34a8bbfc404237345c416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26e1b5455bd47c1b488604a66feb697', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:22.210721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:22.211165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:22.211452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:22.212142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:22.215672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:22.261603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 968/1919] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:22 2022] POST /identity/v3/projects => generated 353 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:22.267540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:22.285695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ec76e-25c4-45d1-ba03-9252c26d3115 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:22.380591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed9ddb9e7e34a8bbfc404237345c416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26e1b5455bd47c1b488604a66feb697', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:22.381319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:22.381725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:22.382108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:22.382886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:22.383254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:22.385165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:22.399165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 952/1920] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:22.470977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:22.486605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35d27e7-045e-43ac-af1d-d99dfa6b6705 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:22.502146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7ed9ddb9e7e34a8bbfc404237345c416', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b26e1b5455bd47c1b488604a66feb697', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:22.503497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:22.504325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:22.504927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] PATH_INFO: `/v3/projects/c19da82d0b58404e882d313840299117` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:22.507551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorizing `identity:delete_project(project_id=c19da82d0b58404e882d313840299117)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:22.509793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:22.596729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c19da82d0b58404e882d313840299117', 'request_id': 'req-754d707f-3477-42ec-9248-e73814552341'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:22.612767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c19da82d0b58404e882d313840299117', 'request_id': 'req-754d707f-3477-42ec-9248-e73814552341'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:22.628247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c19da82d0b58404e882d313840299117', 'request_id': 'req-754d707f-3477-42ec-9248-e73814552341'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:22.659744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] The token cache is being invalidate because project c19da82d0b58404e882d313840299117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:22.660968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:22.663005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:22.666796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 969/1921] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:22 2022] DELETE /identity/v3/projects/c19da82d0b58404e882d313840299117 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:23.095946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:23.099115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:23.111723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:23.113751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:23.181814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:23.221661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c198a14b-911f-42e7-85a0-fb8cd7097573 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:23.225301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754d707f-3477-42ec-9248-e73814552341 tempest-VolumeQuotaClassesTest-223845396 tempest-VolumeQuotaClassesTest-223845396-project-admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:23.230747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:23.231437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:23.231644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:23.231644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:23.231919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] PATH_INFO: `/v3/users/41b2ea939e8043fbb7ae10d2bf085476` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:23.232540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:23.232988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:23.233150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b2ea939e8043fbb7ae10d2bf085476)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:23.233496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] PATH_INFO: `/v3/users/cfcfdca81d3b4bb39b8bfd6b2019844f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:23.235188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:23.237620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=cfcfdca81d3b4bb39b8bfd6b2019844f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:23.242450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:23.294568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b2ea939e8043fbb7ae10d2bf085476'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.298094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b2ea939e8043fbb7ae10d2bf085476'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.298787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b2ea939e8043fbb7ae10d2bf085476'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.308837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b2ea939e8043fbb7ae10d2bf085476'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.311369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:23.331540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cfcfdca81d3b4bb39b8bfd6b2019844f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.332875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cfcfdca81d3b4bb39b8bfd6b2019844f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.333872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cfcfdca81d3b4bb39b8bfd6b2019844f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.341119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1e68cb8-cf53-4af3-9d80-1fae5f56e9f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b2ea939e8043fbb7ae10d2bf085476'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.345658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cfcfdca81d3b4bb39b8bfd6b2019844f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.362691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6f26bda6-0347-47ec-9cc5-650e5408b9cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cfcfdca81d3b4bb39b8bfd6b2019844f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:23.367220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 970/1922] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:23 2022] DELETE /identity/v3/users/41b2ea939e8043fbb7ae10d2bf085476 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:23.370121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:23.370849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:23.370849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:23.395162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:23.396054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:23.412577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 953/1923] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:23 2022] DELETE /identity/v3/users/cfcfdca81d3b4bb39b8bfd6b2019844f => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:23.415473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:23.415811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:23.416088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:23.452067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:23.454314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:23.476279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 971/1924] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:23.492258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:23.498336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:23.584411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 954/1925] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:23.600682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:23.603800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:23.613141 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd1895f-4b47-45c5-a0f3-cc74da41c0fa None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:23.710250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:23.711066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:23.711465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:23.711834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:23.722168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:23.724029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:23.724612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ff7b5-8af8-4452-883f-47c0780a54cb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:23.730527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:23.731423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:23.731635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:23.731990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:23.739631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 972/1926] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:23.744055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:23.749498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:23.753466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:23.773723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 955/1927] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:24.355178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:24.365242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:24.392149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ce1c95-360d-4fc9-ba48-c0bf07a10086 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:24.423598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:24.426609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:24.428537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:24.429296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] PATH_INFO: `/v3/projects/a2f0d79da8744ed9a741c91025c87b48` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:24.437382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f0d79da8744ed9a741c91025c87b48)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:24.439906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:24.498077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:24.559036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f0d79da8744ed9a741c91025c87b48', 'request_id': 'req-ad066fbe-f235-423c-9268-197becf244cb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:24.578438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f0d79da8744ed9a741c91025c87b48', 'request_id': 'req-ad066fbe-f235-423c-9268-197becf244cb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:24.594808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:24.598627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:24.599723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f0d79da8744ed9a741c91025c87b48', 'request_id': 'req-ad066fbe-f235-423c-9268-197becf244cb'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:24.630335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] The token cache is being invalidate because project a2f0d79da8744ed9a741c91025c87b48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:24.630918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:24.632039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:24.635943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 973/1928] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:24 2022] DELETE /identity/v3/projects/a2f0d79da8744ed9a741c91025c87b48 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:24.641699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:24.643316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:24.712715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304afda3-5701-42cd-9af5-fcbcdac9764b service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:24.859346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad066fbe-f235-423c-9268-197becf244cb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:24.869424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:24.870987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:24.871648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:24.872451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] PATH_INFO: `/v3/users/7ed9ddb9e7e34a8bbfc404237345c416` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:24.874920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed9ddb9e7e34a8bbfc404237345c416)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:24.880253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:24.937903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:24.939415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:24.939747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:24.939990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] PATH_INFO: `/v3/projects/755d5b6317d54cdbaa87a1a306d55891` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:24.943714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=755d5b6317d54cdbaa87a1a306d55891)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:24.947382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:25.009203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed9ddb9e7e34a8bbfc404237345c416'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.009932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed9ddb9e7e34a8bbfc404237345c416'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.010614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed9ddb9e7e34a8bbfc404237345c416'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.022401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed9ddb9e7e34a8bbfc404237345c416'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.088719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed9ddb9e7e34a8bbfc404237345c416'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.100574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '755d5b6317d54cdbaa87a1a306d55891', 'request_id': 'req-b5a0dc1f-e648-4836-b00a-73b2253dba27'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.109986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '755d5b6317d54cdbaa87a1a306d55891', 'request_id': 'req-b5a0dc1f-e648-4836-b00a-73b2253dba27'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.122993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '755d5b6317d54cdbaa87a1a306d55891', 'request_id': 'req-b5a0dc1f-e648-4836-b00a-73b2253dba27'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.132419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 974/1929] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:24 2022] DELETE /identity/v3/users/7ed9ddb9e7e34a8bbfc404237345c416 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:25.151186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] The token cache is being invalidate because project 755d5b6317d54cdbaa87a1a306d55891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:25.151799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.152820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.155615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 956/1930] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:24 2022] DELETE /identity/v3/projects/755d5b6317d54cdbaa87a1a306d55891 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:25.164264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:25.168138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:25.234686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:25.280675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1465f562-642e-4fac-8e88-ccab5965e726 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:25.287949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:25.288723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:25.289223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:25.289600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] PATH_INFO: `/v3/users/4a88e21c9db249b7a5b3b09c6bdf9a1b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:25.291605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a88e21c9db249b7a5b3b09c6bdf9a1b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:25.294639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:25.397711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a88e21c9db249b7a5b3b09c6bdf9a1b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.398576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a88e21c9db249b7a5b3b09c6bdf9a1b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.399293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a88e21c9db249b7a5b3b09c6bdf9a1b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.409046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a88e21c9db249b7a5b3b09c6bdf9a1b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.431115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a88e21c9db249b7a5b3b09c6bdf9a1b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:25.455386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 975/1931] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:25 2022] DELETE /identity/v3/users/4a88e21c9db249b7a5b3b09c6bdf9a1b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:26.689693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:26.692068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:26.758861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:26.761667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:26.844085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b091491-1368-441b-8c61-5dd9e06fdca9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:26.849665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:26.850362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:26.850636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:26.850907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] PATH_INFO: `/v3/projects/199422ae7e7442f189207ccedf5be394` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:26.852232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] RBAC: Authorizing `identity:delete_project(project_id=199422ae7e7442f189207ccedf5be394)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:26.857197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:26.899129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0dc1f-e648-4836-b00a-73b2253dba27 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:26.905272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:26.906021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:26.906405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:26.906753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] PATH_INFO: `/v3/projects/b26e1b5455bd47c1b488604a66feb697` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:26.909080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b26e1b5455bd47c1b488604a66feb697)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:26.911712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:26.967458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '199422ae7e7442f189207ccedf5be394', 'request_id': 'req-d2e7f348-75fa-4069-a52a-aea2aa52964c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:26.972142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:26.977221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:26.980330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '199422ae7e7442f189207ccedf5be394', 'request_id': 'req-d2e7f348-75fa-4069-a52a-aea2aa52964c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:26.998396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '199422ae7e7442f189207ccedf5be394', 'request_id': 'req-d2e7f348-75fa-4069-a52a-aea2aa52964c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.022497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b26e1b5455bd47c1b488604a66feb697', 'request_id': 'req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.037544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] The token cache is being invalidate because project 199422ae7e7442f189207ccedf5be394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:27.037826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.042855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d2e7f348-75fa-4069-a52a-aea2aa52964c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.043851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b26e1b5455bd47c1b488604a66feb697', 'request_id': 'req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.052110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 976/1932] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:26 2022] DELETE /identity/v3/projects/199422ae7e7442f189207ccedf5be394 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:27.056196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d4b0-b25e-47ff-8d24-23ab533135d8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.056196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d4b0-b25e-47ff-8d24-23ab533135d8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.056196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b36d4b0-b25e-47ff-8d24-23ab533135d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.059694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b26e1b5455bd47c1b488604a66feb697', 'request_id': 'req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.086067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] The token cache is being invalidate because project b26e1b5455bd47c1b488604a66feb697 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:27.086667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.087374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5f894446-d1bb-4abd-b4a9-eef8e6de2d59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.095997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 957/1933] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:26 2022] DELETE /identity/v3/projects/b26e1b5455bd47c1b488604a66feb697 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:27.099477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d5e8-061d-4a14-bf07-f566bfa753e7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.100086 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d5e8-061d-4a14-bf07-f566bfa753e7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.100649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d5e8-061d-4a14-bf07-f566bfa753e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.106564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2b36d4b0-b25e-47ff-8d24-23ab533135d8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:27.109647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2b36d4b0-b25e-47ff-8d24-23ab533135d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.139609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9e26d5e8-061d-4a14-bf07-f566bfa753e7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:27.142359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9e26d5e8-061d-4a14-bf07-f566bfa753e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.234906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 977/1934] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:27.245540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.246056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.246520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.258250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 958/1935] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:27.270488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.270977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.271401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.289634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:27.292816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.300633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:27.302004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.312165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 959/1936] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:27.319402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.331577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac6889-b985-46a4-95a7-97890a1c6260 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.333713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 978/1937] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:27.339259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.341480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.342881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.343813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.344550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.346069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:27.346657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.350876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.357091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f205c-d522-48bc-aaf7-957486d101f6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.366504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.367427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.367903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.368391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.370353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:27.370543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.372374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 960/1938] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:27.376795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.379887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.395034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 979/1939] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:27.398464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fff348-a448-4fd4-8d35-d6d8c830c17b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.401481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.410643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.411918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.412417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.413017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.414629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.417077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e4ab61-a3ff-42bd-b74c-92e00d44c361 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.418863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.423806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.424509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.424932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.425791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.425937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.428175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.468537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 980/1940] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:27.472735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.505669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 961/1941] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:27.507461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.613801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1fac68-bdd9-4410-8218-b000a8d207b6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.648014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9493f9f1-8424-4565-a5ba-2b6dc56428df admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.658108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.661762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.661762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.661762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.662391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.665848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.689257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:27.705376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 962/1942] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:27.707702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.710228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.721918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:27.723519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.725196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.725196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.725670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.726999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.731690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.733535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a5fc07-3d49-4564-a689-b1b1e6283abb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.742779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.743758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.744205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.744645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] PATH_INFO: `/v3/users/fb7c120737664d55b4fb9d2ac1492291` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.746212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb7c120737664d55b4fb9d2ac1492291)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.748680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.773816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 981/1943] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/users => generated 294 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:27.773816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.786063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60716783-8319-4453-9cc0-520205863bdb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.792934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.794159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.794665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.795131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.796259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.799339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.834290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb7c120737664d55b4fb9d2ac1492291'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.835452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb7c120737664d55b4fb9d2ac1492291'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.836554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb7c120737664d55b4fb9d2ac1492291'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.837321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 982/1944] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:27.839371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.844670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.849011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb7c120737664d55b4fb9d2ac1492291'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.858796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb7c120737664d55b4fb9d2ac1492291'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:27.871001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ae840-2069-4aaa-bb14-92fc46c65705 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:27.880065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924d5a57-862a-4204-8e8e-0c3944faba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:27.881039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5a57-862a-4204-8e8e-0c3944faba69 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.881497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5a57-862a-4204-8e8e-0c3944faba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.881951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924d5a57-862a-4204-8e8e-0c3944faba69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.883142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5a57-862a-4204-8e8e-0c3944faba69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:27.885952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924d5a57-862a-4204-8e8e-0c3944faba69 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:27.892613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 963/1945] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:27 2022] DELETE /identity/v3/users/fb7c120737664d55b4fb9d2ac1492291 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:27.892613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:27.892613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:27.902732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 983/1946] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:27.904989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:27.905486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:27.905963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:27.946024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:27.950385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:28.025462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a214bc-77a3-47d9-a717-497333a4dae7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.031625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.032255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.032669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.032948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.033602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.035362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.039623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 984/1947] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:28.043836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.049269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 964/1948] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:27 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:28.057606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.077291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19ce53c-f6be-4738-96da-213b17df3368 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.080586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f213e6a-3537-46d7-afb5-d9d162247fa1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.083252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.084130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.084500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.084880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.085718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.087548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.090004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.090969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.091452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.091922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] PATH_INFO: `/v3/projects/4fc099617a00402094e9c3519c87ef6a/users/e903537814c34356a853081786809aa3/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.104816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 985/1949] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:28.106411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.109534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:28.128967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fc099617a00402094e9c3519c87ef6a, user_id=e903537814c34356a853081786809aa3, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.133988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.161901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 965/1950] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:28 2022] PUT /identity/v3/projects/4fc099617a00402094e9c3519c87ef6a/users/e903537814c34356a853081786809aa3/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:28.163109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.198326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9665f452-2c39-4aea-a670-7c3ced5603b3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.234432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:28.267253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f41ae5-832d-4718-97c5-7f02e9c1e2b0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.276286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ad84-cb79-4854-bb3c-fbbbd6170d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.277186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ad84-cb79-4854-bb3c-fbbbd6170d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.277637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ad84-cb79-4854-bb3c-fbbbd6170d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.278075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ad84-cb79-4854-bb3c-fbbbd6170d7f admin admin] PATH_INFO: `/v3/projects/6e5556f0eb884bbe8937e8dd837f0d07/users/2cd03bc067684a9bbd5481f3a54d89ba/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.302462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ad84-cb79-4854-bb3c-fbbbd6170d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e5556f0eb884bbe8937e8dd837f0d07, user_id=2cd03bc067684a9bbd5481f3a54d89ba, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.305244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ad84-cb79-4854-bb3c-fbbbd6170d7f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.310690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a27300-115e-4df6-916c-c7e39907d88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.312046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a27300-115e-4df6-916c-c7e39907d88d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.312653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a27300-115e-4df6-916c-c7e39907d88d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.313404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a27300-115e-4df6-916c-c7e39907d88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.322909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 966/1951] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:28 2022] PUT /identity/v3/projects/6e5556f0eb884bbe8937e8dd837f0d07/users/2cd03bc067684a9bbd5481f3a54d89ba/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:28.324051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.324405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.324737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.344563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a27300-115e-4df6-916c-c7e39907d88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.347796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a27300-115e-4df6-916c-c7e39907d88d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.369752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] MFA Rules not processed for user `e903537814c34356a853081786809aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:28.371037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:28.374564 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 986/1952] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:28.379260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.379560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.379800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.401281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:28.437408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] MFA Rules not processed for user `2cd03bc067684a9bbd5481f3a54d89ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:28.439315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:28.462464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 967/1953] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:28.469989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.592602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 987/1954] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:28.601197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.642297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d19756-b2f1-43eb-a165-c27e0156f44a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.655620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.656261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.656546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.656963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.657620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.659622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.746304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 968/1955] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:28 2022] POST /identity/v3/projects => generated 377 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:28.751088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.760963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d481d-075d-415b-98c2-9905aa23af58 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.851154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:28.898571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdcec85d-8d4f-4c2c-8732-a2fcff9c2977 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.905231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.906592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.907004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.907355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.908141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.910335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.917067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.918308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.918896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.919461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.922089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.926213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.938588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 969/1956] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:28 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:28.944687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:28.960437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b8323-e2ca-4faf-b4d8-1353eeaac971 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:28.967692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:28.969013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:28.969535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:28.970088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:28.971758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:28.973355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:28.987223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 988/1957] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:28 2022] POST /identity/v3/projects => generated 389 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:28.999451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.000779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 970/1958] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:28 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:29.008322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.023054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b77ea9-53bd-4d8c-913f-be1f4ec0d5c0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.066957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:29.116198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df9edf-7798-493c-8821-179e4c06e669 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.123561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.124672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.125045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.125413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.125951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.129494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.157993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.158654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.158986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.160253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.160253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.161838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.168435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 989/1959] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:28 2022] POST /identity/v3/users => generated 301 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:29.169831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.172727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:29.181230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 971/1960] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:29.181230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.193188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7021146-ad81-4190-8672-51f56957ece6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.193851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5488064-5612-4176-8822-8870dd1ab00a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.199920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.200652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.201006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.201357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.202082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.203393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.203974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.204395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.204784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.205370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] PATH_INFO: `/v3/projects/cfd1a32424914d6a90e6f9e3719dfeb3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.208765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfd1a32424914d6a90e6f9e3719dfeb3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.211317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.218227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 972/1961] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:29.219045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.230524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a402540b-5f6b-498b-b272-1b251fdaadaa admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.236418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.237822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.237822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.237822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] PATH_INFO: `/v3/projects/2a158cf5c3e24220b21d60ac1d05cf7e/users/d4fcf17147cb4af2b73d1d014342daa7/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.256280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a158cf5c3e24220b21d60ac1d05cf7e, user_id=d4fcf17147cb4af2b73d1d014342daa7, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.259180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.304127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 973/1962] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:29 2022] PUT /identity/v3/projects/2a158cf5c3e24220b21d60ac1d05cf7e/users/d4fcf17147cb4af2b73d1d014342daa7/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:29.305660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.336320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfd1a32424914d6a90e6f9e3719dfeb3', 'request_id': 'req-690327ae-efd0-4be6-a6e0-977fb4b12619'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.347499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfd1a32424914d6a90e6f9e3719dfeb3', 'request_id': 'req-690327ae-efd0-4be6-a6e0-977fb4b12619'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.358477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfd1a32424914d6a90e6f9e3719dfeb3', 'request_id': 'req-690327ae-efd0-4be6-a6e0-977fb4b12619'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.388583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] The token cache is being invalidate because project cfd1a32424914d6a90e6f9e3719dfeb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:29.388940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.389683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-690327ae-efd0-4be6-a6e0-977fb4b12619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.395483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 990/1963] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:29 2022] DELETE /identity/v3/projects/cfd1a32424914d6a90e6f9e3719dfeb3 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:29.397351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.397705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.398014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.416560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6b798f-77bf-462d-bc72-f7ae6428c5e8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.447489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] MFA Rules not processed for user `d4fcf17147cb4af2b73d1d014342daa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:29.449577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:29.477722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:29.542406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.543794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.544430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.545118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] PATH_INFO: `/v3/projects/60ac88476df048fc9bf15d96534fb7c1/users/024269c0a16f411a82b58af59534414a/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.570747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ac88476df048fc9bf15d96534fb7c1, user_id=024269c0a16f411a82b58af59534414a, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.573950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.581096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 991/1964] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:29.581096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.584702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:29.606765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 974/1965] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:29 2022] PUT /identity/v3/projects/60ac88476df048fc9bf15d96534fb7c1/users/024269c0a16f411a82b58af59534414a/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:29.607796 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.612976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:29.621099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ea7f5-a245-4b4b-8425-58758d2682e3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.712635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:29.758293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4051ece-6f81-4112-aa5b-bf54f8930062 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:29.770237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.772465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.772734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.773724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.775454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.780709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.807812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:29.808726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:29.809161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:29.809581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] PATH_INFO: `/v3/users/50df9c8cbc3c4de9b533cc5b1e4f2168` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:29.811608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=50df9c8cbc3c4de9b533cc5b1e4f2168)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:29.815483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:29.860782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 975/1966] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:29 2022] POST /identity/v3/projects => generated 377 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:29.862942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:29.867514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:29.920713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50df9c8cbc3c4de9b533cc5b1e4f2168'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.923395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50df9c8cbc3c4de9b533cc5b1e4f2168'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.925589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50df9c8cbc3c4de9b533cc5b1e4f2168'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.945534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50df9c8cbc3c4de9b533cc5b1e4f2168'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:29.995330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50df9c8cbc3c4de9b533cc5b1e4f2168'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:30.020149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 992/1967] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:29 2022] DELETE /identity/v3/users/50df9c8cbc3c4de9b533cc5b1e4f2168 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:30.022162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.024680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:30.039929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cdf3c8-ac08-431d-aaef-83c6c5f54143 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.178004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:30.200934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.202054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.220483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.222561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.223300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.234365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.236208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c730f3b-74d3-4e7a-80e4-87fc55088e2c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.251948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.257132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.257451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.257731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.258467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.260482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.271438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 976/1968] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:29 2022] GET /identity/v3/roles => generated 1420 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:30.277450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.285464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:30.302597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc7a6d-1e1b-407c-b982-25f53a156e77 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.310869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9bb607-5ca1-4e1c-b59b-226e705b2e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.311695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bb607-5ca1-4e1c-b59b-226e705b2e35 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.312071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bb607-5ca1-4e1c-b59b-226e705b2e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.312440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9bb607-5ca1-4e1c-b59b-226e705b2e35 admin admin] PATH_INFO: `/v3/projects/60ac88476df048fc9bf15d96534fb7c1/users/024269c0a16f411a82b58af59534414a/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.314593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bb607-5ca1-4e1c-b59b-226e705b2e35 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60ac88476df048fc9bf15d96534fb7c1, user_id=024269c0a16f411a82b58af59534414a, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.320886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9bb607-5ca1-4e1c-b59b-226e705b2e35 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.321173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 993/1969] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:30 2022] POST /identity/v3/users => generated 295 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:30.326288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.337141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 977/1970] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:30 2022] PUT /identity/v3/projects/60ac88476df048fc9bf15d96534fb7c1/users/024269c0a16f411a82b58af59534414a/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:30.348016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fff47e4-af24-4da5-beea-66d57e82cc6a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.348915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.349301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.350795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.394995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] MFA Rules not processed for user `024269c0a16f411a82b58af59534414a`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:30.396405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:30.424771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:30.529787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.531861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.533391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.533674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.535429 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.537203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.560614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 994/1971] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:30.566604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.607596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb593620-434e-427e-9625-18789fd06aa4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.623407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 978/1972] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:30 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:30.627907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.629716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.630124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.630552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] PATH_INFO: `/v3/projects/7d91818d2b7b48cd8511488bf5349e29/users/0409d13506f547988a6640dfd2c7925c/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.637976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.639613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:30.675415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d91818d2b7b48cd8511488bf5349e29, user_id=0409d13506f547988a6640dfd2c7925c, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.681082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.714449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 995/1973] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:30 2022] PUT /identity/v3/projects/7d91818d2b7b48cd8511488bf5349e29/users/0409d13506f547988a6640dfd2c7925c/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:30.716599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.768968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334d9c4-60b8-4cb3-be78-c368293186af None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.839636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebd59fd-c713-4335-b4d8-5c11ee350b16 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.845710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.848545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.848993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.849442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.850437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.852669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.867678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 996/1974] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:30 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:30.871649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:30.886291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.886291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.886291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.886291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.897404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547d140-94a4-4dad-b856-ed8ae1a789e2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:30.917719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e9641-2dbf-4766-85e8-1bf1e556062f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:30.921460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9641-2dbf-4766-85e8-1bf1e556062f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.922475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9641-2dbf-4766-85e8-1bf1e556062f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.924042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e9641-2dbf-4766-85e8-1bf1e556062f admin admin] PATH_INFO: `/v3/projects/7d91818d2b7b48cd8511488bf5349e29/users/0409d13506f547988a6640dfd2c7925c/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:30.927060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e9641-2dbf-4766-85e8-1bf1e556062f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d91818d2b7b48cd8511488bf5349e29, user_id=0409d13506f547988a6640dfd2c7925c, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:30.935422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e9641-2dbf-4766-85e8-1bf1e556062f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:30.954448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:30.974311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 997/1975] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:30 2022] PUT /identity/v3/projects/7d91818d2b7b48cd8511488bf5349e29/users/0409d13506f547988a6640dfd2c7925c/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:30.986943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:30.989722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:30.994225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:31.070710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] MFA Rules not processed for user `0409d13506f547988a6640dfd2c7925c`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:31.070710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:31.083072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:31.084839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:31.153207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 979/1976] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:30 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:31.304386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 998/1977] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:31.311340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:31.313083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:31.332612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd456c08-a8ed-4df3-a54b-5fe265579358 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:31.358232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:31.400898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:31.447090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:31.447738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:31.451350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:31.451849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:31.482875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:31.486385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:31.497935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae71708-eace-40c3-8e41-da20c201ee02 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:31.528988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 980/1978] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:31.542520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:31.548218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:31.550015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:31.552070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] PATH_INFO: `/v3/users/2cd03bc067684a9bbd5481f3a54d89ba` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:31.556661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cd03bc067684a9bbd5481f3a54d89ba)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:31.571377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:31.619055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:31.633580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5d977f-ee75-4fb1-b57c-9f8b60890edf service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:31.647367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:31.648158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:31.648542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:31.648920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:31.651753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:31.713022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cd03bc067684a9bbd5481f3a54d89ba'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:31.714150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cd03bc067684a9bbd5481f3a54d89ba'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:31.715044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cd03bc067684a9bbd5481f3a54d89ba'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:31.727095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cd03bc067684a9bbd5481f3a54d89ba'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:31.747776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8a143c8a-0d38-4f61-88b4-567fab11a9bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cd03bc067684a9bbd5481f3a54d89ba'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:31.775043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 999/1979] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:31 2022] DELETE /identity/v3/users/2cd03bc067684a9bbd5481f3a54d89ba => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:31.779631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:31.780036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:31.780462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:31.791065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:31.794207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:31.809313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:31.810535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:31.826730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 981/1980] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:31.944115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1000/1981] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:32.021864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:32.023681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:32.235329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc0b40f-8201-4134-ae1b-9b7502058c90 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:32.324727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:32.441954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbfdca9-21cd-40af-8698-adc5fee9b4a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:32.443384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdca9-21cd-40af-8698-adc5fee9b4a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:32.444043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdca9-21cd-40af-8698-adc5fee9b4a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:32.444771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbfdca9-21cd-40af-8698-adc5fee9b4a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:32.469799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:32.472966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:32.476946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfdca9-21cd-40af-8698-adc5fee9b4a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:32.495680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbfdca9-21cd-40af-8698-adc5fee9b4a0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:32.505922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 982/1982] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:32.526658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f158a076-3a99-41bd-8237-61bbf19376f6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:32.532004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:32.533198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:32.533198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:32.533361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] PATH_INFO: `/v3/projects/ff65f6271f5142debc630b8ff631492e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:32.535580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff65f6271f5142debc630b8ff631492e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:32.537802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:32.625273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:32.657574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff65f6271f5142debc630b8ff631492e', 'request_id': 'req-d288385b-fe71-474f-93f7-e7409aa85e47'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:32.669845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff65f6271f5142debc630b8ff631492e', 'request_id': 'req-d288385b-fe71-474f-93f7-e7409aa85e47'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:32.683017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff65f6271f5142debc630b8ff631492e', 'request_id': 'req-d288385b-fe71-474f-93f7-e7409aa85e47'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:32.722519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] The token cache is being invalidate because project ff65f6271f5142debc630b8ff631492e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:32.723300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:32.724197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d288385b-fe71-474f-93f7-e7409aa85e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:32.727363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1001/1983] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:32 2022] DELETE /identity/v3/projects/ff65f6271f5142debc630b8ff631492e => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:32.733068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:32.733519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:32.733920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:32.780287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:32.788792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:32.903609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 983/1984] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:32.913834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:32.914349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:32.914723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:32.947456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:32.949882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:32.966771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1002/1985] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:32.972815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:32.992656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb91c4d-5223-4231-a979-d7730d334353 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.001255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.002230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.002230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.002230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.002836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:33.003147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.005406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.026088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 984/1986] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:33.028127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.056176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd37028-d0ff-4aba-ab12-035f38226a10 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.063922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.066635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:33.067093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.068112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.068561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.069043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.070018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.072941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.100868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d6ea8-bd79-485f-a857-b452ea49224e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.119609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.120463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.120876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.121066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1003/1987] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:33 2022] POST /identity/v3/projects => generated 365 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:33.121324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] PATH_INFO: `/v3/projects/6e5556f0eb884bbe8937e8dd837f0d07` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.122745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e5556f0eb884bbe8937e8dd837f0d07)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.125309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.134156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.207797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e5556f0eb884bbe8937e8dd837f0d07', 'request_id': 'req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.213393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e5556f0eb884bbe8937e8dd837f0d07', 'request_id': 'req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.226798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e5556f0eb884bbe8937e8dd837f0d07', 'request_id': 'req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.246692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] The token cache is being invalidate because project 6e5556f0eb884bbe8937e8dd837f0d07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:33.247022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.247719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.248672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ff9c7a-ba24-4631-84c3-9f896e549e40 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.249732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 985/1988] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:33 2022] DELETE /identity/v3/projects/6e5556f0eb884bbe8937e8dd837f0d07 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:33.253938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.255368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:33.301549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:33.330953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.332039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.332395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.332669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.333312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.335537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.350763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f854fcd-0b43-4db7-90fa-8fa08cd77b35 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.360020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.360779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.361249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.361718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] PATH_INFO: `/v3/users/024269c0a16f411a82b58af59534414a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.363554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1004/1989] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:33 2022] POST /identity/v3/users => generated 283 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:33.367210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=024269c0a16f411a82b58af59534414a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.367903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.369220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.372396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:33.389190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ffe131-75ea-4893-b643-c02d1109446c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.395199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.395893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.396489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.396956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.397823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.399643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.412218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1005/1990] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:33.416416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.427060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8800ac-55ae-4f3f-a511-8d7486f1c20c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.432211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '024269c0a16f411a82b58af59534414a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.432832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.433525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '024269c0a16f411a82b58af59534414a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.433836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.434339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.434809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.434923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '024269c0a16f411a82b58af59534414a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.436031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.439175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.446091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '024269c0a16f411a82b58af59534414a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.457397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1006/1991] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:33 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:33.459387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-815d5b85-a2f4-48d9-974c-96146156daaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '024269c0a16f411a82b58af59534414a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:33.463190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.465179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:33.485455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 986/1992] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:33 2022] DELETE /identity/v3/users/024269c0a16f411a82b58af59534414a => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:33.487209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b18232-94c3-4d61-9d36-f376d886aef8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.492972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:33.580862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.582357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.583046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.583046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] PATH_INFO: `/v3/projects/62838eb074ce4113b553624867b25111/users/746c63106cc84f20919b3ad24631a30f/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.632806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=62838eb074ce4113b553624867b25111, user_id=746c63106cc84f20919b3ad24631a30f, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.637627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:33.666892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1007/1993] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:33 2022] PUT /identity/v3/projects/62838eb074ce4113b553624867b25111/users/746c63106cc84f20919b3ad24631a30f/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:33.668827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.669115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.669391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.704714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] MFA Rules not processed for user `746c63106cc84f20919b3ad24631a30f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:33.706141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:33.827532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 987/1994] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:33.837226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:33.969272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065c05ee-6788-4421-ba1c-78c896db7a91 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:33.979380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:33.980884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:33.981391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:33.981912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:33.983073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:33.989147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.063638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.065363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1008/1995] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:33 2022] POST /identity/v3/projects => generated 367 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:34.080329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcd204-a49b-47c8-a0ad-576ab04de684 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.098438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.102060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.158793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:34.209228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5eff492-7650-4f42-b26c-0bf7129dc0fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.210010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff492-7650-4f42-b26c-0bf7129dc0fb admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.210326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff492-7650-4f42-b26c-0bf7129dc0fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.210671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5eff492-7650-4f42-b26c-0bf7129dc0fb admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.213239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff492-7650-4f42-b26c-0bf7129dc0fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.219749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5eff492-7650-4f42-b26c-0bf7129dc0fb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.284062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 988/1996] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:34 2022] POST /identity/v3/users => generated 290 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:34.284062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8cdbc-d466-4f1e-9f4d-5efdc4a18aad None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.284062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8cdbc-d466-4f1e-9f4d-5efdc4a18aad None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.284062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8cdbc-d466-4f1e-9f4d-5efdc4a18aad None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.290914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 989/1997] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:37:34 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:37:34.291856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8cdbc-d466-4f1e-9f4d-5efdc4a18aad None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.319909 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8cdbc-d466-4f1e-9f4d-5efdc4a18aad None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.329135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1233fea5-0f27-4b81-af72-89956530be99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.330013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233fea5-0f27-4b81-af72-89956530be99 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.330657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233fea5-0f27-4b81-af72-89956530be99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.331144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1233fea5-0f27-4b81-af72-89956530be99 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.331892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233fea5-0f27-4b81-af72-89956530be99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.333567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1233fea5-0f27-4b81-af72-89956530be99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.343316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73940750-7398-4150-84d1-f1dbd6340c5f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.347744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 990/1998] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:34.350016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.350169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.350516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.362449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.362449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.362449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.362449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] PATH_INFO: `/v3/projects/60ac88476df048fc9bf15d96534fb7c1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.365513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60ac88476df048fc9bf15d96534fb7c1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.373773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.375019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:34.378074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.403489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:34.480338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60ac88476df048fc9bf15d96534fb7c1', 'request_id': 'req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:34.486555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60ac88476df048fc9bf15d96534fb7c1', 'request_id': 'req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:34.496829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60ac88476df048fc9bf15d96534fb7c1', 'request_id': 'req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:34.510801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 991/1999] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:34.511981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.516634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] The token cache is being invalidate because project 60ac88476df048fc9bf15d96534fb7c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:34.517064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:34.518166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1c4c65e8-94e1-4267-95d3-71f8f896eb0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:34.520457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1009/2000] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:34 2022] DELETE /identity/v3/projects/60ac88476df048fc9bf15d96534fb7c1 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:34.525343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.525829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.526159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.528448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad93b99-5559-41e7-b0d2-5866519ffde5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.553607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:34.555565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.607947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e68ceed-6f0b-48f1-9663-e528916b4571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.608508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68ceed-6f0b-48f1-9663-e528916b4571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.608782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68ceed-6f0b-48f1-9663-e528916b4571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.609059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e68ceed-6f0b-48f1-9663-e528916b4571 admin admin] PATH_INFO: `/v3/projects/ba8eee2198a941e1b6dd4ae71c6611c9/users/5f8e4af34e7c414994c4372506d99f69/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.626201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68ceed-6f0b-48f1-9663-e528916b4571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8eee2198a941e1b6dd4ae71c6611c9, user_id=5f8e4af34e7c414994c4372506d99f69, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.629751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e68ceed-6f0b-48f1-9663-e528916b4571 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.634314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1010/2001] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:34.635183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.636683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.651531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 992/2002] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:34 2022] PUT /identity/v3/projects/ba8eee2198a941e1b6dd4ae71c6611c9/users/5f8e4af34e7c414994c4372506d99f69/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:34.657276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.657571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.657813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.694380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:34.696335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.735169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:34.747126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db0f158-9076-43e2-a8c1-5a02d5f9d2f9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.779069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 993/2003] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:34.780620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.783516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.803631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71a20b-769f-463e-85d5-439c7e31bf84 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.811711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.812335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.812790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.813226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.816963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.819932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.845870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 994/2004] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:34.846128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:34.846320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.861439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36167d18-2732-4f10-89f3-d966aa559960 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.882334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.882914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.883515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.883687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.883865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.884020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.884168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.884920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.885844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:34.886152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:34.886693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.888780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.904614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 995/2005] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:34.905715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.907175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.919458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607c9160-808d-494c-8a2d-bc71ae8006ce admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:34.920914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.930961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee792015-0e6b-4d5b-a7af-6e0569db4e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:34.930961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee792015-0e6b-4d5b-a7af-6e0569db4e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:34.930961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee792015-0e6b-4d5b-a7af-6e0569db4e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:34.930961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee792015-0e6b-4d5b-a7af-6e0569db4e21 admin admin] PATH_INFO: `/v3/projects/ba8eee2198a941e1b6dd4ae71c6611c9/users/5f8e4af34e7c414994c4372506d99f69/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:34.932092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee792015-0e6b-4d5b-a7af-6e0569db4e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba8eee2198a941e1b6dd4ae71c6611c9, user_id=5f8e4af34e7c414994c4372506d99f69, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:34.945082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee792015-0e6b-4d5b-a7af-6e0569db4e21 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:34.960424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1011/2006] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:34.962234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:34.987968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 996/2007] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:34 2022] PUT /identity/v3/projects/ba8eee2198a941e1b6dd4ae71c6611c9/users/5f8e4af34e7c414994c4372506d99f69/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:35.005295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.005586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.005977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.012158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eba065b-efc5-4e68-a575-3b056b62e4a2 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.077081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] MFA Rules not processed for user `5f8e4af34e7c414994c4372506d99f69`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:35.078385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:35.177048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:35.177705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.177993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.178315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.179308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:35.181038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:35.248418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1012/2008] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:34 2022] POST /identity/v3/projects => generated 379 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:35.254523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.270665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 997/2009] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:35 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:35.281403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.283264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:35.373941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7faa86-a922-4e53-8f03-9aeb9b49dd6b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.401425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d259adbf-c980-4b01-b5d0-32faab3c3502 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.407577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:35.408881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.409163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.409437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.410853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:35.414609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:35.423453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:35.448356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1013/2010] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:35 2022] POST /identity/v3/users => generated 290 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:35.454644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.470974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd9794d-279a-4615-9dae-f4503cf0eaa0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.494835 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:35.495345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.495684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.495967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.496622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:35.498273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:35.507064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:35.507693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.507978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.508284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.519980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1014/2011] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:35.525350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.538773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e2ea2a-9b26-4098-a228-c53fd16867ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.545149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:35.545992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.546536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.547207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.547942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:35.551026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:35.574807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1015/2012] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:35.580683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.603349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5731b-796e-4600-9b61-fb7b86fae999 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.610910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5e811-9ba2-4b53-96a6-f15a402f998c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:35.612203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5e811-9ba2-4b53-96a6-f15a402f998c admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.613053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5e811-9ba2-4b53-96a6-f15a402f998c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.613613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5e811-9ba2-4b53-96a6-f15a402f998c admin admin] PATH_INFO: `/v3/projects/38e579a0294b46b69dfd61331d25f517/users/1b91518e3b6c4a62b6add2a36c16eafa/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.654111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5e811-9ba2-4b53-96a6-f15a402f998c admin admin] RBAC: Authorizing `identity:create_grant(project_id=38e579a0294b46b69dfd61331d25f517, user_id=1b91518e3b6c4a62b6add2a36c16eafa, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:35.659683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef5e811-9ba2-4b53-96a6-f15a402f998c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:35.680446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:35.694691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1016/2013] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:35 2022] PUT /identity/v3/projects/38e579a0294b46b69dfd61331d25f517/users/1b91518e3b6c4a62b6add2a36c16eafa/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:35.715929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:35.716476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:35.716813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:35.752807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:35.755431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:35.771446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] MFA Rules not processed for user `1b91518e3b6c4a62b6add2a36c16eafa`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:35.772865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:35.775168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 998/2014] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:35.881759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1017/2015] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:35.886580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.899002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9397d49-92fe-4c95-b15d-fb420dd33b90 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:35.912976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:35.999358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.000056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.000344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.000671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.001375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.003809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.020420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d899526a-bebc-4339-a4aa-228b5f911dba None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.025880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.026833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.027157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.027632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.029604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:36.055556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 999/2016] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:35 2022] POST /identity/v3/projects => generated 381 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:36.060214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.182349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.184084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.190866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28caffc7-59a4-4155-a2c9-68c4ec01951a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.198056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.198308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1018/2017] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:36.199045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.199341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.199619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.200452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.202486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.247872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1000/2018] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:36 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:36.252169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.264759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac2b799-9b23-452a-b92e-8335a802706a service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.282060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.282060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.282480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.282627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.283286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.285963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.298345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1019/2019] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:36.303373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.312585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:36.325121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5731e6cb-cf10-4d78-89d8-c7e8d5ee169d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.334512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.335652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.336099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.336566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] PATH_INFO: `/v3/projects/2157e6bedfe24bfeaa5c0ed0a1475b42/users/64bbc1b6da014fa39dea2dfffa074bf2/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.381144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2157e6bedfe24bfeaa5c0ed0a1475b42, user_id=64bbc1b6da014fa39dea2dfffa074bf2, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.384500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.411542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1001/2020] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:36 2022] PUT /identity/v3/projects/2157e6bedfe24bfeaa5c0ed0a1475b42/users/64bbc1b6da014fa39dea2dfffa074bf2/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:36.413015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.433546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5fd93-593a-4c97-ab6d-ab5e08609194 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.486899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:36.526532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23fc9c-5166-45ce-82a7-c3f100da1914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.527606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23fc9c-5166-45ce-82a7-c3f100da1914 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.528019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23fc9c-5166-45ce-82a7-c3f100da1914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.528317 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23fc9c-5166-45ce-82a7-c3f100da1914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.529173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23fc9c-5166-45ce-82a7-c3f100da1914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.531372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23fc9c-5166-45ce-82a7-c3f100da1914 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.547960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1020/2021] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:36 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:36.549258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.582051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f1e4f-f69a-41d3-939f-e066f6fe68b4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.599075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.599312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.599565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.599887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] PATH_INFO: `/v3/projects/2157e6bedfe24bfeaa5c0ed0a1475b42/users/64bbc1b6da014fa39dea2dfffa074bf2/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.602023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2157e6bedfe24bfeaa5c0ed0a1475b42, user_id=64bbc1b6da014fa39dea2dfffa074bf2, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.604833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.622760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1002/2022] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:36 2022] PUT /identity/v3/projects/2157e6bedfe24bfeaa5c0ed0a1475b42/users/64bbc1b6da014fa39dea2dfffa074bf2/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:36.636169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.636463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.636737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.670717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] MFA Rules not processed for user `64bbc1b6da014fa39dea2dfffa074bf2`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:36.671200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:36.773768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1021/2023] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:36.787176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.791293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:36.876612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e10207-c0ae-45d7-980b-94efb9d8bdb7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:36.881343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf67668-b195-4e34-9453-c684db2e4667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:36.881970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf67668-b195-4e34-9453-c684db2e4667 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:36.882353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf67668-b195-4e34-9453-c684db2e4667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:36.882905 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf67668-b195-4e34-9453-c684db2e4667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:36.894949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf67668-b195-4e34-9453-c684db2e4667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:36.896559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf67668-b195-4e34-9453-c684db2e4667 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:36.921040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1003/2024] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:36.949971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:36.951759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.046131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150155ea-5c2e-4e94-9c87-1913cb00fd96 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:37.054705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:37.056329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.056815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.057264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] PATH_INFO: `/v3/users/e903537814c34356a853081786809aa3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:37.058642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e903537814c34356a853081786809aa3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:37.060980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:37.157274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e903537814c34356a853081786809aa3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.158101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e903537814c34356a853081786809aa3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.158933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e903537814c34356a853081786809aa3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.172256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e903537814c34356a853081786809aa3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.174931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:37.197091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e903537814c34356a853081786809aa3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.226231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1022/2025] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:36 2022] DELETE /identity/v3/users/e903537814c34356a853081786809aa3 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:37.229492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.229958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.230209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:37.251619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:37.252854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.341901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1004/2026] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:37.353402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:37.354884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.436881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703882ce-d749-41d9-bf10-d2702650e3cb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:37.441970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a863c6-3d1d-47ee-bfdf-3e0b84927d17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:37.442848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a863c6-3d1d-47ee-bfdf-3e0b84927d17 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.444113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a863c6-3d1d-47ee-bfdf-3e0b84927d17 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.445244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a863c6-3d1d-47ee-bfdf-3e0b84927d17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:37.457094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a863c6-3d1d-47ee-bfdf-3e0b84927d17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:37.458743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a863c6-3d1d-47ee-bfdf-3e0b84927d17 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:37.469345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1023/2027] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:37.508428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:37.511277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.532261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9311b5cb-9dd5-4d29-8699-72bbcaa1ecde None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:37.542908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:37.543565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.544195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.544469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] PATH_INFO: `/v3/users/1b91518e3b6c4a62b6add2a36c16eafa` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:37.546553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b91518e3b6c4a62b6add2a36c16eafa)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:37.550725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:37.668498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b91518e3b6c4a62b6add2a36c16eafa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.669231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b91518e3b6c4a62b6add2a36c16eafa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.669944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b91518e3b6c4a62b6add2a36c16eafa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.693162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b91518e3b6c4a62b6add2a36c16eafa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.705029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b91518e3b6c4a62b6add2a36c16eafa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:37.715978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:37.765542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1005/2028] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:37 2022] DELETE /identity/v3/users/1b91518e3b6c4a62b6add2a36c16eafa => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:37.773397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.773769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.774289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:37.797896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:37.799255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.893452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1024/2029] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:37.904488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:37.906832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.948016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:37.953049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:37.976853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6990d6-a8bf-48bd-a6f5-d145a2acdded None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:37.981868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d47154-3765-4a4a-bb1a-a8d6180c5ed7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:37.984353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:37.985009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.985269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.985497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] PATH_INFO: `/v3/projects/4fc099617a00402094e9c3519c87ef6a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:37.986961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fc099617a00402094e9c3519c87ef6a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:37.988916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:37.989976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:37.990260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:37.990755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:37.991016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:38.013619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:38.015170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:38.028091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1006/2030] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:38.070831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fc099617a00402094e9c3519c87ef6a', 'request_id': 'req-897c1034-ee4b-4b31-9c67-b6332996677d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.081451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fc099617a00402094e9c3519c87ef6a', 'request_id': 'req-897c1034-ee4b-4b31-9c67-b6332996677d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.091906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fc099617a00402094e9c3519c87ef6a', 'request_id': 'req-897c1034-ee4b-4b31-9c67-b6332996677d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.114701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:38.125061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] The token cache is being invalidate because project 4fc099617a00402094e9c3519c87ef6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:38.127482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.133193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.137794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1025/2031] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:37 2022] DELETE /identity/v3/projects/4fc099617a00402094e9c3519c87ef6a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:38.142800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:38.175024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f2abc4-ab46-442b-b6b6-e1b68c59962c service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:38.348819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:38.349750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:38.350560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:38.350837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] PATH_INFO: `/v3/users/d4fcf17147cb4af2b73d1d014342daa7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:38.353103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4fcf17147cb4af2b73d1d014342daa7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:38.355923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:38.436750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4fcf17147cb4af2b73d1d014342daa7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.438150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4fcf17147cb4af2b73d1d014342daa7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.439377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4fcf17147cb4af2b73d1d014342daa7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.452453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4fcf17147cb4af2b73d1d014342daa7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.463998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4fcf17147cb4af2b73d1d014342daa7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.500582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1007/2032] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:38 2022] DELETE /identity/v3/users/d4fcf17147cb4af2b73d1d014342daa7 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:38.583814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:38.585507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:38.601164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c1034-ee4b-4b31-9c67-b6332996677d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:38.673234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:38.673833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:38.674290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:38.674558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] PATH_INFO: `/v3/projects/38e579a0294b46b69dfd61331d25f517` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:38.675596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38e579a0294b46b69dfd61331d25f517)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:38.677572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:38.748849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38e579a0294b46b69dfd61331d25f517', 'request_id': 'req-5fbec7da-ab91-48ad-827d-f7134a6bf009'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.757017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38e579a0294b46b69dfd61331d25f517', 'request_id': 'req-5fbec7da-ab91-48ad-827d-f7134a6bf009'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.766224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38e579a0294b46b69dfd61331d25f517', 'request_id': 'req-5fbec7da-ab91-48ad-827d-f7134a6bf009'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.783070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] The token cache is being invalidate because project 38e579a0294b46b69dfd61331d25f517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:38.783546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.791127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:38.793073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1026/2033] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:38 2022] DELETE /identity/v3/projects/38e579a0294b46b69dfd61331d25f517 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:38.811218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:38.813961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:38.827956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:38.840280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f905ab-b193-4a93-a620-3d817678ff30 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:39.008065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:39.009720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:39.010543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:39.010955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] PATH_INFO: `/v3/users/64bbc1b6da014fa39dea2dfffa074bf2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:39.012484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64bbc1b6da014fa39dea2dfffa074bf2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:39.014766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:39.129047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64bbc1b6da014fa39dea2dfffa074bf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.129896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64bbc1b6da014fa39dea2dfffa074bf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.130830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64bbc1b6da014fa39dea2dfffa074bf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.146872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64bbc1b6da014fa39dea2dfffa074bf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.157561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64bbc1b6da014fa39dea2dfffa074bf2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.193297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1008/2034] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:38 2022] DELETE /identity/v3/users/64bbc1b6da014fa39dea2dfffa074bf2 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:39.275821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:39.282444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:39.318426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbec7da-ab91-48ad-827d-f7134a6bf009 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:39.432752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:39.433630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:39.433974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:39.434392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] PATH_INFO: `/v3/projects/2a158cf5c3e24220b21d60ac1d05cf7e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:39.436797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a158cf5c3e24220b21d60ac1d05cf7e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:39.439613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:39.498778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:39.524765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a158cf5c3e24220b21d60ac1d05cf7e', 'request_id': 'req-01497b49-cd07-49bd-96e4-341720837cd3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.551438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a158cf5c3e24220b21d60ac1d05cf7e', 'request_id': 'req-01497b49-cd07-49bd-96e4-341720837cd3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.579717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a158cf5c3e24220b21d60ac1d05cf7e', 'request_id': 'req-01497b49-cd07-49bd-96e4-341720837cd3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.603791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] The token cache is being invalidate because project 2a158cf5c3e24220b21d60ac1d05cf7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:39.604626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.605428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.607366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1027/2035] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:39 2022] DELETE /identity/v3/projects/2a158cf5c3e24220b21d60ac1d05cf7e => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:39.611483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:39.613627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:39.630226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966eb9f4-e4e0-4e6e-9689-b206551c5b82 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:39.707985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:39.708728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:39.709019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:39.709286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] PATH_INFO: `/v3/users/0409d13506f547988a6640dfd2c7925c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:39.710554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0409d13506f547988a6640dfd2c7925c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:39.712540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:39.741786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:39.783743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0409d13506f547988a6640dfd2c7925c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.784544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0409d13506f547988a6640dfd2c7925c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.785932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0409d13506f547988a6640dfd2c7925c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.798013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0409d13506f547988a6640dfd2c7925c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.807960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8a9e2509-a10c-4940-ab19-a1b69763ff80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0409d13506f547988a6640dfd2c7925c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:39.832255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:39.833958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:39.848549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01497b49-cd07-49bd-96e4-341720837cd3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:39.850936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1009/2036] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:39 2022] DELETE /identity/v3/users/0409d13506f547988a6640dfd2c7925c => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:39.948843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:39.949637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:39.949986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:39.950310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] PATH_INFO: `/v3/projects/2157e6bedfe24bfeaa5c0ed0a1475b42` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:39.951980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2157e6bedfe24bfeaa5c0ed0a1475b42)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:39.954512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:40.043509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2157e6bedfe24bfeaa5c0ed0a1475b42', 'request_id': 'req-d145884a-ac26-4370-90cc-99e89bbb13b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.058974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2157e6bedfe24bfeaa5c0ed0a1475b42', 'request_id': 'req-d145884a-ac26-4370-90cc-99e89bbb13b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.069535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2157e6bedfe24bfeaa5c0ed0a1475b42', 'request_id': 'req-d145884a-ac26-4370-90cc-99e89bbb13b2'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.108647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] The token cache is being invalidate because project 2157e6bedfe24bfeaa5c0ed0a1475b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:40.108966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.109672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.118700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1028/2037] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:39 2022] DELETE /identity/v3/projects/2157e6bedfe24bfeaa5c0ed0a1475b42 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:40.120756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc2305-447e-4cba-86ad-f0eecf4428a2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.121046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc2305-447e-4cba-86ad-f0eecf4428a2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.121363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bc2305-447e-4cba-86ad-f0eecf4428a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.147780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-96bc2305-447e-4cba-86ad-f0eecf4428a2 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:40.149811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-96bc2305-447e-4cba-86ad-f0eecf4428a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:40.220581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:40.225162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:40.252096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1010/2038] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:40.253775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.254048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.254356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.299035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:40.309822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:40.314044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:40.323395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d145884a-ac26-4370-90cc-99e89bbb13b2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:40.334069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:40.334341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1011/2039] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:40.335628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.336238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.336695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] PATH_INFO: `/v3/projects/7d91818d2b7b48cd8511488bf5349e29` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.338247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d91818d2b7b48cd8511488bf5349e29)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:40.339530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:40.341225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:40.400895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594a0a59-67ad-4021-833f-a44199e78bca None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:40.432196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d91818d2b7b48cd8511488bf5349e29', 'request_id': 'req-33be8102-6c98-4152-89a8-9396063f7348'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.441106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d91818d2b7b48cd8511488bf5349e29', 'request_id': 'req-33be8102-6c98-4152-89a8-9396063f7348'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.450425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d91818d2b7b48cd8511488bf5349e29', 'request_id': 'req-33be8102-6c98-4152-89a8-9396063f7348'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.519912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] The token cache is being invalidate because project 7d91818d2b7b48cd8511488bf5349e29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:40.520622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.521344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-33be8102-6c98-4152-89a8-9396063f7348 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:40.525698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1029/2040] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:40 2022] DELETE /identity/v3/projects/7d91818d2b7b48cd8511488bf5349e29 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:40.536413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897d268-c629-420c-b41a-23f0969f5388 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.537008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897d268-c629-420c-b41a-23f0969f5388 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.537551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a897d268-c629-420c-b41a-23f0969f5388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.590767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a897d268-c629-420c-b41a-23f0969f5388 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:40.592454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a897d268-c629-420c-b41a-23f0969f5388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:40.604168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:40.606960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.607090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1030/2041] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:40.609462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.609766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.610508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:40.612047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:40.620327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:40.624739 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.625023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.625298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.651352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1012/2042] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:40.655920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:40.698473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:40.723320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:40.724637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:40.749967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1031/2043] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:40.759908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:40.762012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f61c3d0-053a-487a-b970-a154c1064cd6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:40.777232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:40.777826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d30ca9-dce2-4eea-a1bd-b6ba1791352c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:40.781107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.782034 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.782813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.784061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:40.789164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:40.789369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:40.790510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.790810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.791314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.792415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:40.793030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:40.797109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:40.833950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1032/2044] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:37:40.835274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:40.857195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35039bba-749a-4c01-8145-0cc58c9e6548 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:40.862906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:40.863563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:40.863845 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:40.864119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:40.864756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:40.867126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:40.891726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1013/2045] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/projects => generated 381 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:40.904398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:40.926931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:40.931083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1033/2046] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:40.943968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.064994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7fe9-76bc-48cf-913c-a3078388af23 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.119666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5f8cde-ce0d-4a4f-8112-97cbe6569ccc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.127558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.128221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.128504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.128781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.129475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.131579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.157193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1034/2047] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/users => generated 288 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:41.162496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.202786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.203575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.204033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.204474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.205220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.208306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.224478 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d52d7-907e-43ea-b35e-ed456dd90dad admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.242140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.244551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.244846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.245106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.245767 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.248272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.258420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1014/2048] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:40 2022] POST /identity/v3/users => generated 291 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:41.261639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1035/2049] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:41.267184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.267516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.279752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81e902c-53f7-4e9f-b5c0-eadf833fe44a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.280418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b74a6a-d3dc-430e-98f3-2ed6e33e9c0b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.285940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.286902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.287183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.287458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.287708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.288108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.288726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.289189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.289785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.289957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.290949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.292217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:41.294869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.310398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1015/2050] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:41.315503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.327075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1036/2051] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:41.328261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.329962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3252f9f-b5ce-41bb-9efc-0bef567f5984 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.336927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.337953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.338380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.338657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.339318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.341089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.342877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd308009-83d4-4471-9e88-a37d774b96ba admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.354773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1016/2052] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:41.355554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8289bbd-eb8b-4799-bc86-8eb17f42bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.357517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289bbd-eb8b-4799-bc86-8eb17f42bb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.360053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.360182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289bbd-eb8b-4799-bc86-8eb17f42bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.363721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8289bbd-eb8b-4799-bc86-8eb17f42bb3b admin admin] PATH_INFO: `/v3/projects/6aff53d47cb04b22a049f87f04410f46/users/5d6a15d3b8554c19bbf378401c3a71ee/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.373572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fd087-1df3-4614-a8d9-d843125cac7f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.380753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2aa02d9-d769-47c5-8411-01eb7ff12b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.382109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa02d9-d769-47c5-8411-01eb7ff12b1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.382609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa02d9-d769-47c5-8411-01eb7ff12b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.383040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa02d9-d769-47c5-8411-01eb7ff12b1a admin admin] PATH_INFO: `/v3/projects/0ef1f3b85db5479a9ec6233fb6752551/users/b5056c165ce84ac3943d1ddc8a32555f/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.405819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa02d9-d769-47c5-8411-01eb7ff12b1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef1f3b85db5479a9ec6233fb6752551, user_id=b5056c165ce84ac3943d1ddc8a32555f, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.409343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2aa02d9-d769-47c5-8411-01eb7ff12b1a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.411790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8289bbd-eb8b-4799-bc86-8eb17f42bb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6aff53d47cb04b22a049f87f04410f46, user_id=5d6a15d3b8554c19bbf378401c3a71ee, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.414635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8289bbd-eb8b-4799-bc86-8eb17f42bb3b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.443481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1017/2053] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:41 2022] PUT /identity/v3/projects/0ef1f3b85db5479a9ec6233fb6752551/users/b5056c165ce84ac3943d1ddc8a32555f/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:41.454712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1037/2054] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:41 2022] PUT /identity/v3/projects/6aff53d47cb04b22a049f87f04410f46/users/5d6a15d3b8554c19bbf378401c3a71ee/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:41.461675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.462081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.462782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.469084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.469653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.470133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.541096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] MFA Rules not processed for user `b5056c165ce84ac3943d1ddc8a32555f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:41.543023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:41.555253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] MFA Rules not processed for user `5d6a15d3b8554c19bbf378401c3a71ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:41.557632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:41.647177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:41.709296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1018/2055] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:41.716682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.735157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1038/2056] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:41.766347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:41.772136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:41.934275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cca17b7-d70e-4fff-bb76-d5e926898bb7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.949992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.951473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.951970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.952438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.953534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.959362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea906ca-a29b-438f-a2e2-f86e30dfb7e3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:41.962275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:41.973658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:41.974433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:41.974780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:41.975118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:41.991864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:41.993600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.019033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1039/2057] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:42.042510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1019/2058] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:41 2022] POST /identity/v3/projects => generated 383 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:42.044180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.060592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff0d68f-f776-4a9f-8651-39dec234aa6e service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.173618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.174274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.174570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.175738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.176161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.178168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.190550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:42.210788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.212805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1040/2059] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:42 2022] POST /identity/v3/users => generated 298 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:42.221704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae915e46-10ee-4e05-ba67-b16b7efce869 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.228517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.228995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.229262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.229642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.230223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.232109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.258981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1020/2060] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:42.261866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.273912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9fcbf0-25a3-4fe2-ba4e-985d478e284c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.286073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.291256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.291543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.291784 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] PATH_INFO: `/v3/projects/347ec452897d40e8b5962d04a274ef3a/users/d34ea779486448729c65a75231b81609/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.315285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347ec452897d40e8b5962d04a274ef3a, user_id=d34ea779486448729c65a75231b81609, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.318473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.357839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1041/2061] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:42 2022] PUT /identity/v3/projects/347ec452897d40e8b5962d04a274ef3a/users/d34ea779486448729c65a75231b81609/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:42.359415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.376677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2444d43-bd5a-4de7-bb6e-bc7dabbdeb0a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.488834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8dab63-cb94-4f8e-8eb3-834012d73473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.493324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dab63-cb94-4f8e-8eb3-834012d73473 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.493922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dab63-cb94-4f8e-8eb3-834012d73473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.494592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8dab63-cb94-4f8e-8eb3-834012d73473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.495779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dab63-cb94-4f8e-8eb3-834012d73473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.499047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8dab63-cb94-4f8e-8eb3-834012d73473 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.514813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1021/2062] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:42 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:42.520256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.542678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34416a85-d61f-4fa1-8372-d02d884399d5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.550003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.550768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.550994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.551270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] PATH_INFO: `/v3/projects/347ec452897d40e8b5962d04a274ef3a/users/d34ea779486448729c65a75231b81609/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.553241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=347ec452897d40e8b5962d04a274ef3a, user_id=d34ea779486448729c65a75231b81609, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.556126 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.575418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1042/2063] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:42 2022] PUT /identity/v3/projects/347ec452897d40e8b5962d04a274ef3a/users/d34ea779486448729c65a75231b81609/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:42.587080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.587480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.587480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.623189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] MFA Rules not processed for user `d34ea779486448729c65a75231b81609`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:42.624456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:42.692106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:42.727632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1022/2064] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:42 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:42.740697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.742834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:42.802551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf2a6f8-d906-4019-a89c-f8eac9072717 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.806460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.807076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.807360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.807631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.817118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.818592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:42.828687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1043/2065] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:37:42.905236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:42.916455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1a43ab-0301-460c-a6f1-ac2c4047bf95 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:42.920634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:42.921354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:42.921633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:42.922164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:42.989390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:42.991589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:43.004318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1023/2066] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:42 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:43.436601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:43.570536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b08c1d-9ce4-455d-adb2-d4a77e615699 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:43.592326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:43.593049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:43.593326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:43.594283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] PATH_INFO: `/v3/users/b5056c165ce84ac3943d1ddc8a32555f` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:43.595714 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5056c165ce84ac3943d1ddc8a32555f)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:43.601526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:43.630858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:43.632393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:43.668808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d5f0b6-3acf-4ddb-adc0-52da4b5fd11d service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:43.686994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:43.687823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:43.688220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:43.688598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] PATH_INFO: `/v3/users/746c63106cc84f20919b3ad24631a30f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:43.693085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] RBAC: Authorizing `identity:delete_user(user_id=746c63106cc84f20919b3ad24631a30f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:43.695876 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:43.754695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5056c165ce84ac3943d1ddc8a32555f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.755863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5056c165ce84ac3943d1ddc8a32555f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.756983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5056c165ce84ac3943d1ddc8a32555f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.784685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5056c165ce84ac3943d1ddc8a32555f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.795275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:43.820085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-62f931ec-1623-4997-840f-509f30f47936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5056c165ce84ac3943d1ddc8a32555f'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.883882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1044/2067] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:43 2022] DELETE /identity/v3/users/b5056c165ce84ac3943d1ddc8a32555f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:43.884215 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '746c63106cc84f20919b3ad24631a30f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.887692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '746c63106cc84f20919b3ad24631a30f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.888257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '746c63106cc84f20919b3ad24631a30f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.889682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:43.890267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:43.890781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:43.923255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '746c63106cc84f20919b3ad24631a30f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.943822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8c6aa765-3d1e-46f4-a062-c69adbc3889e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '746c63106cc84f20919b3ad24631a30f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:43.946493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:43.948022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:43.992066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1024/2068] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:43 2022] DELETE /identity/v3/users/746c63106cc84f20919b3ad24631a30f => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:43.995741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:43.996724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:43.997502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:44.044024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:44.047513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:44.148617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1045/2069] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:44.157006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:44.163054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:44.165425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:44.194560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1025/2070] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:44.202897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:44.205035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:44.266143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a7bba4-bd8f-4d67-a876-c793c582730d None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:44.341005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039a129a-2e09-4d92-8e12-084f46892fd7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:44.345667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:44.346540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:44.346854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:44.347712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:44.360194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:44.363147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:44.365391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:44.366445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:44.366926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:44.367198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:44.377412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:44.378951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:44.383627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1026/2071] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:44.390161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1046/2072] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:44.934284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:44.935864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:44.953652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808eebdb-c74c-4bc7-8ffa-232134c59038 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:44.959721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:44.960424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:44.960788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:44.961131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] PATH_INFO: `/v3/projects/0ef1f3b85db5479a9ec6233fb6752551` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:44.962393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef1f3b85db5479a9ec6233fb6752551)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:44.965102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:45.038769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef1f3b85db5479a9ec6233fb6752551', 'request_id': 'req-0eaa6f11-93aa-40fb-960f-dab9482e11f5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.049586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef1f3b85db5479a9ec6233fb6752551', 'request_id': 'req-0eaa6f11-93aa-40fb-960f-dab9482e11f5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.068036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef1f3b85db5479a9ec6233fb6752551', 'request_id': 'req-0eaa6f11-93aa-40fb-960f-dab9482e11f5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.078275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:45.079795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:45.082319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] The token cache is being invalidate because project 0ef1f3b85db5479a9ec6233fb6752551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:45.082656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.083414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.084911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1027/2073] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:44 2022] DELETE /identity/v3/projects/0ef1f3b85db5479a9ec6233fb6752551 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:45.090657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:45.092337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:45.097336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f268ed1-1c5a-45d4-83cf-e05ad7541007 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:45.178269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:45.201238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaa6f11-93aa-40fb-960f-dab9482e11f5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:45.206624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:45.207674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:45.207940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:45.208213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] PATH_INFO: `/v3/users/d34ea779486448729c65a75231b81609` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:45.209573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d34ea779486448729c65a75231b81609)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:45.211589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:45.223643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:45.224944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:45.225456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:45.226005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] PATH_INFO: `/v3/projects/62838eb074ce4113b553624867b25111` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:45.230602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=62838eb074ce4113b553624867b25111)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:45.234236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:45.246520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:45.285365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd34ea779486448729c65a75231b81609'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.286049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd34ea779486448729c65a75231b81609'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.286719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd34ea779486448729c65a75231b81609'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.305338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd34ea779486448729c65a75231b81609'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.319718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62838eb074ce4113b553624867b25111', 'request_id': 'req-49c73fff-5814-4705-8b12-0c39c2de6226'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.334643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62838eb074ce4113b553624867b25111', 'request_id': 'req-49c73fff-5814-4705-8b12-0c39c2de6226'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.335951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd34ea779486448729c65a75231b81609'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.344177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62838eb074ce4113b553624867b25111', 'request_id': 'req-49c73fff-5814-4705-8b12-0c39c2de6226'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.365171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] The token cache is being invalidate because project 62838eb074ce4113b553624867b25111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:45.365461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.366311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.371030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1047/2074] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:45 2022] DELETE /identity/v3/projects/62838eb074ce4113b553624867b25111 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:45.372263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:45.373761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:45.382938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1028/2075] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:45 2022] DELETE /identity/v3/users/d34ea779486448729c65a75231b81609 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:45.476697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c73fff-5814-4705-8b12-0c39c2de6226 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:45.553663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:45.553663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:45.554498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:45.554498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] PATH_INFO: `/v3/users/5f8e4af34e7c414994c4372506d99f69` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:45.555322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f8e4af34e7c414994c4372506d99f69)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:45.557366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:45.592058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:45.649109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f8e4af34e7c414994c4372506d99f69'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.650069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f8e4af34e7c414994c4372506d99f69'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.650852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f8e4af34e7c414994c4372506d99f69'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.665615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f8e4af34e7c414994c4372506d99f69'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.689260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f8e4af34e7c414994c4372506d99f69'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:45.707353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1048/2076] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:37:45 2022] DELETE /identity/v3/users/5f8e4af34e7c414994c4372506d99f69 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:45.798098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:45.801807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:45.835385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c881466b-77a6-4051-b706-f5a784ff344c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:45.957240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:45.958357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:45.958829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:45.959340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] PATH_INFO: `/v3/projects/347ec452897d40e8b5962d04a274ef3a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:45.961701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=347ec452897d40e8b5962d04a274ef3a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:45.965198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:46.068214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '347ec452897d40e8b5962d04a274ef3a', 'request_id': 'req-106cde26-4749-4d51-8bb0-f044fb49a813'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.076297 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '347ec452897d40e8b5962d04a274ef3a', 'request_id': 'req-106cde26-4749-4d51-8bb0-f044fb49a813'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.093252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '347ec452897d40e8b5962d04a274ef3a', 'request_id': 'req-106cde26-4749-4d51-8bb0-f044fb49a813'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.120283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] The token cache is being invalidate because project 347ec452897d40e8b5962d04a274ef3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:46.120911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.121846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-106cde26-4749-4d51-8bb0-f044fb49a813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.124669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1029/2077] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:45 2022] DELETE /identity/v3/projects/347ec452897d40e8b5962d04a274ef3a => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:46.274997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:46.277572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:46.302590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f1558c-c11a-4429-a77d-bce6d595e5fe admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:46.399796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:46.399796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.400022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.400449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] PATH_INFO: `/v3/projects/ba8eee2198a941e1b6dd4ae71c6611c9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.402535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba8eee2198a941e1b6dd4ae71c6611c9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:46.405948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:46.482706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba8eee2198a941e1b6dd4ae71c6611c9', 'request_id': 'req-468e9751-681f-43eb-9842-fd7e5cfcd731'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.491396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba8eee2198a941e1b6dd4ae71c6611c9', 'request_id': 'req-468e9751-681f-43eb-9842-fd7e5cfcd731'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.499952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba8eee2198a941e1b6dd4ae71c6611c9', 'request_id': 'req-468e9751-681f-43eb-9842-fd7e5cfcd731'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.521639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] The token cache is being invalidate because project ba8eee2198a941e1b6dd4ae71c6611c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:37:46.522293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.523444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-468e9751-681f-43eb-9842-fd7e5cfcd731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:37:46.528234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1049/2078] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:37:46 2022] DELETE /identity/v3/projects/ba8eee2198a941e1b6dd4ae71c6611c9 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:46.533562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.534083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.535277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.571599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:46.573002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:46.607687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:46.655815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1030/2079] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:46.664538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.664814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.665088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.686314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:46.687542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:46.695907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1050/2080] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:46.700839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:46.714305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9379a45a-355f-416e-a1f1-081c7879597e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:46.719843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:46.720462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.720741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.721014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.721669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:37:46.722692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:46.724557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:46.739313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1031/2081] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:37:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:37:46.743916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:46.753915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49444df0-61bd-4938-8b77-1c6ae8610751 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:46.759135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:46.759743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.760052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.760330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.760961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:46.762942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:46.785716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=15,keystone:SELECT=82,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:46.796167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1051/2082] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:46 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:46.800964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:46.813343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9dddc2-54dc-443d-9d6d-827733c7eace admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:46.880494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:46.881105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.881380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.881651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.882323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:46.884276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:46.910032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1032/2083] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:46 2022] POST /identity/v3/users => generated 288 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:37:46.914472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:46.931987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906debd-52fc-4b6c-be05-fc831156098d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:46.937532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:46.938321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.938736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.939078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.940318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:46.942119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:46.954632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1052/2084] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:46.960453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:46.973425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182e0fc-f80a-4c1f-9979-0bdf0b23b1de admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:46.984037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e11d33-53df-4e30-a91b-7fe0f9dfbade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:46.985619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e11d33-53df-4e30-a91b-7fe0f9dfbade admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:46.985737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e11d33-53df-4e30-a91b-7fe0f9dfbade admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:46.986138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e11d33-53df-4e30-a91b-7fe0f9dfbade admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:46.987469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e11d33-53df-4e30-a91b-7fe0f9dfbade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:46.990631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e11d33-53df-4e30-a91b-7fe0f9dfbade admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.013717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1033/2085] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:47.021158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.041702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1308b848-deed-4234-8077-9aef451da006 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.060288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.061711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.062087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.062486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] PATH_INFO: `/v3/projects/177ef97bd13f4051bc003135f45a9308/users/6383ea80e9084d0ab3f01197901a6156/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.086497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=177ef97bd13f4051bc003135f45a9308, user_id=6383ea80e9084d0ab3f01197901a6156, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.095144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.112901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1053/2086] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:47 2022] PUT /identity/v3/projects/177ef97bd13f4051bc003135f45a9308/users/6383ea80e9084d0ab3f01197901a6156/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:47.129390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.129789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.130177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.179175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] MFA Rules not processed for user `6383ea80e9084d0ab3f01197901a6156`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:47.180425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:47.267907 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1034/2087] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:37:47.273851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.352763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83985434-37d9-4626-8f46-5aa9205e1a9f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.358773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.359387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.359666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.359933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.360561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.362554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.400112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1054/2088] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:37:47 2022] POST /identity/v3/projects => generated 373 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:47.404477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.416893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf362cc-3345-445a-a52a-762bc5a3c90a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.445794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:47.512739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.513358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.513634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.513938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.514783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.516734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.544032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1035/2089] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:37:47 2022] POST /identity/v3/users => generated 293 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:37:47.549413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.562097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf61200-30fa-4689-845b-a7b0127e7323 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.567527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.568139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.568417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.568681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.569329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.570983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.583746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1055/2090] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:37:47.587736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.599776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be276d35-6f14-499a-80b8-8e73851bf5c1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.605275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.605963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.606855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.606855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] PATH_INFO: `/v3/projects/1dba211171ad4091b1fce5abf2aa1353/users/b95f6c637ffb4835b983ecfa534527f9/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.624627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dba211171ad4091b1fce5abf2aa1353, user_id=b95f6c637ffb4835b983ecfa534527f9, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.628623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.644692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1036/2091] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:47 2022] PUT /identity/v3/projects/1dba211171ad4091b1fce5abf2aa1353/users/b95f6c637ffb4835b983ecfa534527f9/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:37:47.652228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.663679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df2a3c-52f4-4949-8802-373bc60f977e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.730041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18202229-510c-493e-9e0b-b05a7f80faab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.730855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18202229-510c-493e-9e0b-b05a7f80faab admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.731243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18202229-510c-493e-9e0b-b05a7f80faab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.731675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18202229-510c-493e-9e0b-b05a7f80faab admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.732418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18202229-510c-493e-9e0b-b05a7f80faab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.734091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18202229-510c-493e-9e0b-b05a7f80faab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.747748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1056/2092] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:37:47 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:37:47.753470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.765457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb28b18-07de-48a0-9344-347e2f6fd049 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:47.771388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:47.772241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.772514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.772783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] PATH_INFO: `/v3/projects/1dba211171ad4091b1fce5abf2aa1353/users/b95f6c637ffb4835b983ecfa534527f9/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.774742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dba211171ad4091b1fce5abf2aa1353, user_id=b95f6c637ffb4835b983ecfa534527f9, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:47.777388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:47.788249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1037/2093] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:37:47 2022] PUT /identity/v3/projects/1dba211171ad4091b1fce5abf2aa1353/users/b95f6c637ffb4835b983ecfa534527f9/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:37:47.799075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:47.799610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:47.800079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:47.830397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] MFA Rules not processed for user `b95f6c637ffb4835b983ecfa534527f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:37:47.832257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:47.863417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:47.928880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1057/2094] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:37:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:37:47.941531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:37:47.943364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:37:48.009014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12852c49-9cb6-4809-bd73-ece28220120e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:37:48.014360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e9ca96-8ef4-4b30-b9ac-8daa0327d034 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:37:48.015351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9ca96-8ef4-4b30-b9ac-8daa0327d034 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:37:48.015823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9ca96-8ef4-4b30-b9ac-8daa0327d034 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:37:48.016309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e9ca96-8ef4-4b30-b9ac-8daa0327d034 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:37:48.096529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9ca96-8ef4-4b30-b9ac-8daa0327d034 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:37:48.099219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e9ca96-8ef4-4b30-b9ac-8daa0327d034 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:37:48.115227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1038/2095] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:37:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:37:57.925068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=19 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:37:58.106339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=81,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:01.517284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:01.588586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f780f0-2f6a-45cf-8fa4-0208a877a428 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:01.593176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:01.593805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:01.594085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:01.594395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] PATH_INFO: `/v3/users/6383ea80e9084d0ab3f01197901a6156` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:01.595598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6383ea80e9084d0ab3f01197901a6156)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:01.598144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:01.661914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6383ea80e9084d0ab3f01197901a6156'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:01.662589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6383ea80e9084d0ab3f01197901a6156'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:01.663199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6383ea80e9084d0ab3f01197901a6156'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:01.672807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6383ea80e9084d0ab3f01197901a6156'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:01.688716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6383ea80e9084d0ab3f01197901a6156'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:01.706304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1058/2096] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:01 2022] DELETE /identity/v3/users/6383ea80e9084d0ab3f01197901a6156 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:01.711046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:01.711512 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:01.711788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:01.734070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:01.735420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:01.836298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1039/2097] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:01.851022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:01.853106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:01.941411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f6d16-6497-4da0-903e-076e520089ec admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:01.946488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:01.946930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:01.947215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:01.947745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:01.966100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:01.972190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:01.987180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1059/2098] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:02.291723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:02.293276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:02.309311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e5a61-124c-4bf9-a668-82c5ba7b318f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:02.316774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:02.317275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:02.317542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:02.320432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] PATH_INFO: `/v3/projects/177ef97bd13f4051bc003135f45a9308` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:02.320432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] RBAC: Authorizing `identity:delete_project(project_id=177ef97bd13f4051bc003135f45a9308)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:02.321671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:02.379716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '177ef97bd13f4051bc003135f45a9308', 'request_id': 'req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.386044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '177ef97bd13f4051bc003135f45a9308', 'request_id': 'req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.397502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '177ef97bd13f4051bc003135f45a9308', 'request_id': 'req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.414705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] The token cache is being invalidate because project 177ef97bd13f4051bc003135f45a9308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:02.415018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.415651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.417228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1040/2099] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:02 2022] DELETE /identity/v3/projects/177ef97bd13f4051bc003135f45a9308 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:02.422424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:02.436711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c44d0-12d6-474d-af71-c2e773ea0f1d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:02.482654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:02.565084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:02.565084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:02.565084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:02.565084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] PATH_INFO: `/v3/users/b95f6c637ffb4835b983ecfa534527f9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:02.567085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95f6c637ffb4835b983ecfa534527f9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:02.567085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:02.624059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95f6c637ffb4835b983ecfa534527f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.624544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95f6c637ffb4835b983ecfa534527f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.625273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95f6c637ffb4835b983ecfa534527f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.636763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95f6c637ffb4835b983ecfa534527f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.654484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6e8f5733-5d21-45a2-9b0a-5dc495df8601 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95f6c637ffb4835b983ecfa534527f9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:02.674840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1060/2100] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:02 2022] DELETE /identity/v3/users/b95f6c637ffb4835b983ecfa534527f9 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:03.023104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.025354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:03.041293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4ebd9e-e660-45e6-8d95-8bb9a8308aad admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.155065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.156224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.156664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.157103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] PATH_INFO: `/v3/projects/1dba211171ad4091b1fce5abf2aa1353` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.160480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dba211171ad4091b1fce5abf2aa1353)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.162690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.214592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:03.243974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dba211171ad4091b1fce5abf2aa1353', 'request_id': 'req-059ae44b-5468-4dd0-8e1e-8b35141c18ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:03.258890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dba211171ad4091b1fce5abf2aa1353', 'request_id': 'req-059ae44b-5468-4dd0-8e1e-8b35141c18ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:03.270605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dba211171ad4091b1fce5abf2aa1353', 'request_id': 'req-059ae44b-5468-4dd0-8e1e-8b35141c18ea'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:03.306584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] The token cache is being invalidate because project 1dba211171ad4091b1fce5abf2aa1353 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:03.306888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:03.307571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-059ae44b-5468-4dd0-8e1e-8b35141c18ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:03.309698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1041/2101] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:03 2022] DELETE /identity/v3/projects/1dba211171ad4091b1fce5abf2aa1353 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:03.314698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.314984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.315309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.342519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:03.342519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:03.416605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1061/2102] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:03.428424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.428735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.429015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.457216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:03.458939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:03.467143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1042/2103] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:03.473804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.484015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67761df2-1e93-4f9d-be5d-59381a17c5f9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.488644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.489257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.489534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.489803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.490489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:03.490789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.492389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.506351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1062/2104] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:03.511840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.522842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38157eed-2ea1-49e3-9ee5-716c0b424a45 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.527562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.528183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.528463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.528733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.529373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.531354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.567789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1043/2105] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:03 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:03.569211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.581801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80acff9d-ab41-4f15-b7de-7d82fd0d2dfa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.621626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:03.700383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.701109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.701618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.701960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.702704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.705544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.742231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1063/2106] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:03 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:03.748950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.770877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6e5c42-d858-408b-acc8-d407734cc8ee admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.777361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.778325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.778966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.779399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.780266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.783006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.799208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1044/2107] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:03.803264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.818379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e79afe-88fb-40d8-bc51-4ae46cda4cc1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.831160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6252ac73-c3e7-486a-846c-5c2c4e28e897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.831160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ac73-c3e7-486a-846c-5c2c4e28e897 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.831160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ac73-c3e7-486a-846c-5c2c4e28e897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.831160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6252ac73-c3e7-486a-846c-5c2c4e28e897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.831644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252ac73-c3e7-486a-846c-5c2c4e28e897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.831644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6252ac73-c3e7-486a-846c-5c2c4e28e897 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.844288 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1064/2108] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:03.856668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:03.868021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d3e5-855d-444a-97c2-604c4cb27641 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:03.872878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:03.873551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.873833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.874113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] PATH_INFO: `/v3/projects/2d1f6f4e31a84fefbf0db96db7b73493/users/469f8e051c9e4066b4cac598d281ea5e/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.891471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d1f6f4e31a84fefbf0db96db7b73493, user_id=469f8e051c9e4066b4cac598d281ea5e, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:03.895428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:03.914574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1045/2109] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:03 2022] PUT /identity/v3/projects/2d1f6f4e31a84fefbf0db96db7b73493/users/469f8e051c9e4066b4cac598d281ea5e/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:03.927054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:03.927337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:03.927748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:03.963068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] MFA Rules not processed for user `469f8e051c9e4066b4cac598d281ea5e`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:03.965342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:04.066241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1065/2110] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:04.070971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.180635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1928b6a-1563-433d-b5cd-d84bd5243e7c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.189879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.191134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.191415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.191690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.192335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.194587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:04.234032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1046/2111] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:04 2022] POST /identity/v3/projects => generated 383 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:04.235301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.250178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e6cb70-a4b5-41d7-a5fb-6a0e9f3990f1 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.379191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.379858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.380083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.380349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.380983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.383069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:04.411234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1066/2112] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:04 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:38:04.413011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.425085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db4d7b-645d-4322-811e-3d5859c293cd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.431182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.432101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.432557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.432997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.434139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.437065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:04.458248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1047/2113] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:04.460983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.477393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0255b6-2ae9-4b17-a74d-f688bc740112 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.484384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b2b8cd-8cce-4021-87b5-8bdfd5d10346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.485323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b8cd-8cce-4021-87b5-8bdfd5d10346 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.486753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b8cd-8cce-4021-87b5-8bdfd5d10346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.486753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b2b8cd-8cce-4021-87b5-8bdfd5d10346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.487952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2b8cd-8cce-4021-87b5-8bdfd5d10346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.491476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b2b8cd-8cce-4021-87b5-8bdfd5d10346 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:04.510021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1067/2114] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:04 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:04.514768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.534835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cd968c-8ea5-4fde-a7a4-7c4c7eccde68 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.544483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.545598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.546132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.547637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] PATH_INFO: `/v3/projects/7a662f4100d344b08152953807e07c6d/users/bfda808a2ccf48559233c7bc4d88a17d/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.584132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a662f4100d344b08152953807e07c6d, user_id=bfda808a2ccf48559233c7bc4d88a17d, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.588696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:04.602928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:04.612553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1048/2115] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:04 2022] PUT /identity/v3/projects/7a662f4100d344b08152953807e07c6d/users/bfda808a2ccf48559233c7bc4d88a17d/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:04.624894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.625774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.626267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.672528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] MFA Rules not processed for user `bfda808a2ccf48559233c7bc4d88a17d`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:04.673360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:04.677995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:04.759538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1068/2116] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:04.764675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.843152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4369b4a7-6bc9-40a8-aa03-de68acde6c17 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.848528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.849132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.849406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.849689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.850335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.852303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:04.884344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1049/2117] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:04 2022] POST /identity/v3/projects => generated 385 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:04.888302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:04.900057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d58562-3014-45b7-a1fa-7480772e3273 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:04.971327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:04.972215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:04.972632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:04.973031 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:04.973993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:04.977336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:05.007891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1069/2118] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:04 2022] POST /identity/v3/users => generated 299 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:05.009223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:05.024399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04d0205-2881-4ac8-b3e3-0a9d4264d2d4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:05.034349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:05.035280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:05.035630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:05.035975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:05.036724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:05.038453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:05.052102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1050/2119] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:05.056921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:05.071004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e07f0-0b3e-4b14-8b5a-6f08b08b1032 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:05.076502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:05.077208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:05.077489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:05.077791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] PATH_INFO: `/v3/projects/f73a6547d79645ee88ca08ceb25afea3/users/f5a1756c468d437e86e60a2ea2c72cf1/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:05.101153 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73a6547d79645ee88ca08ceb25afea3, user_id=f5a1756c468d437e86e60a2ea2c72cf1, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:05.103928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:05.125056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1070/2120] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:05 2022] PUT /identity/v3/projects/f73a6547d79645ee88ca08ceb25afea3/users/f5a1756c468d437e86e60a2ea2c72cf1/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:05.129420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:05.147103 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1365f8-59eb-4516-bd61-ec55caf39e80 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:05.245171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d888b-9219-4079-b85e-12d0090f66d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:05.247146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d888b-9219-4079-b85e-12d0090f66d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:05.247146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d888b-9219-4079-b85e-12d0090f66d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:05.247146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d888b-9219-4079-b85e-12d0090f66d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:05.247622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d888b-9219-4079-b85e-12d0090f66d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:05.249673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d888b-9219-4079-b85e-12d0090f66d2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:05.263393 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1051/2121] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:05 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:05.264928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:05.277231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c1f932-22cc-43af-8cd2-57ce7386cd9a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:05.283743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:05.285250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:05.285577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:05.285966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] PATH_INFO: `/v3/projects/f73a6547d79645ee88ca08ceb25afea3/users/f5a1756c468d437e86e60a2ea2c72cf1/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:05.288432 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73a6547d79645ee88ca08ceb25afea3, user_id=f5a1756c468d437e86e60a2ea2c72cf1, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:05.291374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:05.311046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1071/2122] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:05 2022] PUT /identity/v3/projects/f73a6547d79645ee88ca08ceb25afea3/users/f5a1756c468d437e86e60a2ea2c72cf1/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:05.322311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:05.322887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:05.323164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:05.372985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] MFA Rules not processed for user `f5a1756c468d437e86e60a2ea2c72cf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:05.375228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:05.533188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:05.550231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1052/2123] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:05 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:05.559565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:05.561777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:05.652738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d4c6b8-0689-475a-86b7-bb4996aefe5d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:05.658204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:05.658959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:05.659479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:05.659736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:05.670900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:05.740525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:05.741973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:05.751312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1072/2124] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:05 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:38:13.706175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:13.717093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fcd90c-7536-41ad-96ed-8d6fca419cb0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:13.723665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:13.724294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:13.724580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:13.724853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:13.834126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:13.835633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:13.844824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1053/2125] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:14.516217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:14.526348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ee238-2ccb-4845-be48-d70ad93355d7 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:14.534372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18941001-13d5-4884-8d3d-603779467150 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:14.535009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18941001-13d5-4884-8d3d-603779467150 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:14.535290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18941001-13d5-4884-8d3d-603779467150 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:14.535560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18941001-13d5-4884-8d3d-603779467150 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:14.546371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18941001-13d5-4884-8d3d-603779467150 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:14.548069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18941001-13d5-4884-8d3d-603779467150 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:14.564454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1073/2126] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:14 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:17.980895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:17.982462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:18.055548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22cbd8-e749-4528-a896-560c9db4e04c service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:18.061249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:18.061897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:18.063120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:18.063120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] PATH_INFO: `/v3/users/5d6a15d3b8554c19bbf378401c3a71ee` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:18.064020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d6a15d3b8554c19bbf378401c3a71ee)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:18.067165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:18.158005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d6a15d3b8554c19bbf378401c3a71ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:18.159581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d6a15d3b8554c19bbf378401c3a71ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:18.160746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d6a15d3b8554c19bbf378401c3a71ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:18.190862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d6a15d3b8554c19bbf378401c3a71ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:18.200801 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d6a15d3b8554c19bbf378401c3a71ee'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:18.228636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1054/2127] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:17 2022] DELETE /identity/v3/users/5d6a15d3b8554c19bbf378401c3a71ee => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:18.232820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:18.233366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:18.233885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:18.255654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:18.257253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:18.334157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1074/2128] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:18.342828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:18.344605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:18.369368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:18.432152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb4fa2-1e61-4476-b7bd-e3201344a156 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:18.436245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924d2e6-6313-439c-8452-33c615f30e38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:18.436855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924d2e6-6313-439c-8452-33c615f30e38 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:18.437134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924d2e6-6313-439c-8452-33c615f30e38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:18.437407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924d2e6-6313-439c-8452-33c615f30e38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:18.453736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924d2e6-6313-439c-8452-33c615f30e38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:18.453736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924d2e6-6313-439c-8452-33c615f30e38 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:18.466385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1055/2129] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:18.999408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.001233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:19.014550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30111fc4-6150-4575-b7b7-3fe0c31ae03b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.020534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.021254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.021457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.021724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] PATH_INFO: `/v3/projects/6aff53d47cb04b22a049f87f04410f46` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.022963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6aff53d47cb04b22a049f87f04410f46)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.025095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.084933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6aff53d47cb04b22a049f87f04410f46', 'request_id': 'req-a17be77b-1d33-47a7-9b48-5c5c8325e813'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:19.092327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6aff53d47cb04b22a049f87f04410f46', 'request_id': 'req-a17be77b-1d33-47a7-9b48-5c5c8325e813'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:19.101742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6aff53d47cb04b22a049f87f04410f46', 'request_id': 'req-a17be77b-1d33-47a7-9b48-5c5c8325e813'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:19.121274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] The token cache is being invalidate because project 6aff53d47cb04b22a049f87f04410f46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:19.121619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:19.122333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-a17be77b-1d33-47a7-9b48-5c5c8325e813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:19.124575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1075/2130] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:18 2022] DELETE /identity/v3/projects/6aff53d47cb04b22a049f87f04410f46 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:19.130478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.131378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.131378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.154811 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:19.156068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:19.226146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1056/2131] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:19.233860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.234699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.234699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.263073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:19.265215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:19.280918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1076/2132] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:19.285652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.298663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896b041-ece5-45a8-8275-79428834e0b9 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.310439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.311469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.311932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.312415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.313501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:19.313985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.317627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.340311 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1057/2133] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:19.344987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.356062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908e3f8-9292-4c08-838e-3ea200290668 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.361234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.362441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.362725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.363069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.363779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.365846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.387624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:19.410596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1077/2134] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:38:19.414543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.428632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb001308-c9d9-43e4-aff1-cb917f386b9a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.508720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.509777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.510333 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.510830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.511947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.515429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.545123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1058/2135] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/users => generated 288 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:19.550382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.562622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656985e6-f2b4-44a7-ae85-37bff02ba463 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.567892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.568649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.569093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.569450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.570217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.572029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.584054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1078/2136] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:19.589903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.601961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d96194e-1004-44ca-b1e1-b0907edc888e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.606913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a844a1-5063-4244-a746-bd7f2b2bbb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.607516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a844a1-5063-4244-a746-bd7f2b2bbb69 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.607791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a844a1-5063-4244-a746-bd7f2b2bbb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.608056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a844a1-5063-4244-a746-bd7f2b2bbb69 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.608718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a844a1-5063-4244-a746-bd7f2b2bbb69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.610364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a844a1-5063-4244-a746-bd7f2b2bbb69 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.625135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1059/2137] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:19.631412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.644120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5506a13a-b733-4d27-9bca-57d44baf4324 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.651365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.652087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.652462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.652741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] PATH_INFO: `/v3/projects/650b85b0583644bfa2bf7ebde89fb701/users/c2f8fa2b7d1545c8ad9543f2c9a25d84/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.675772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=650b85b0583644bfa2bf7ebde89fb701, user_id=c2f8fa2b7d1545c8ad9543f2c9a25d84, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.678480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.699318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1079/2138] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:19 2022] PUT /identity/v3/projects/650b85b0583644bfa2bf7ebde89fb701/users/c2f8fa2b7d1545c8ad9543f2c9a25d84/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:19.710576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.711187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.711465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.753780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] MFA Rules not processed for user `c2f8fa2b7d1545c8ad9543f2c9a25d84`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:19.755127 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:19.766642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:19.858328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1060/2139] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:19.862415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.889046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:19.890872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:19.952908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0b8ec-6ac1-4302-bfbb-0371ac8499d2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.958738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.959484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.959838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.960321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.961404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.964377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:19.985315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed47126-01c6-4855-9f04-eafc357f959a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:19.991766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:19.993030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:19.993520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:19.994007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] PATH_INFO: `/v3/users/469f8e051c9e4066b4cac598d281ea5e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:19.995056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1080/2140] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:19.996238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=469f8e051c9e4066b4cac598d281ea5e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:19.999081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.000304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.069702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '469f8e051c9e4066b4cac598d281ea5e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.070446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '469f8e051c9e4066b4cac598d281ea5e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.071239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '469f8e051c9e4066b4cac598d281ea5e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.087112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '469f8e051c9e4066b4cac598d281ea5e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.097964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5025e24a-b0af-4417-875c-a70ca577fb40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '469f8e051c9e4066b4cac598d281ea5e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.117357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c942bdd3-c6b9-4125-93e3-e53f6e14ac3b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.120871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1061/2141] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:19 2022] DELETE /identity/v3/users/469f8e051c9e4066b4cac598d281ea5e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:20.125359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.125537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.125712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.126085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.126290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.126473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.126625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.127309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.129237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.147896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:20.149229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:20.155085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1081/2142] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:19 2022] POST /identity/v3/users => generated 287 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:38:20.159188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.160901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:20.177818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c269d3-bfe7-43e3-b8ab-dd4d23859aa5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.182067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:20.224435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:20.229855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1062/2143] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:20.243257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.245381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:20.258676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.259331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.259728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.260086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.260742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.262391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.281357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1082/2144] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:20.283329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.299680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ce7dd-d339-4992-a60d-467349c94e2f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.304563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.305172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.305457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.305729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.306415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.308070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.319700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1083/2145] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:20.322236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927468c2-40e8-4c46-9bf2-3bca7cd7b60b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.324057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.326437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321ff75b-2d95-4d2a-8d09-1699dc49009f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.327072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ff75b-2d95-4d2a-8d09-1699dc49009f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.327348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ff75b-2d95-4d2a-8d09-1699dc49009f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.327618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321ff75b-2d95-4d2a-8d09-1699dc49009f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.340418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58f6fb6-f49f-4dd5-8306-26ee34e76ff0 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.340728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ff75b-2d95-4d2a-8d09-1699dc49009f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.342265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321ff75b-2d95-4d2a-8d09-1699dc49009f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.345591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.346922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.347353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.347643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] PATH_INFO: `/v3/projects/fb21d5e8df3c4a7fa25d092848e7cca8/users/e821c794d7ae4cbd97be7ad4759eabed/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.353708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1063/2146] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:38:20.363764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb21d5e8df3c4a7fa25d092848e7cca8, user_id=e821c794d7ae4cbd97be7ad4759eabed, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.366420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.381057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1084/2147] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:20 2022] PUT /identity/v3/projects/fb21d5e8df3c4a7fa25d092848e7cca8/users/e821c794d7ae4cbd97be7ad4759eabed/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:20.393768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.394095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.394346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.432756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] MFA Rules not processed for user `e821c794d7ae4cbd97be7ad4759eabed`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:20.434043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:20.541968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1064/2148] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:20.544371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.645344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a13981d-1050-424c-bd99-22d603297d02 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.651457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.652085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.652367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.652643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.653283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.655323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.692540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1085/2149] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:20 2022] POST /identity/v3/projects => generated 375 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:20.697009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.709116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24b0b6f-7c75-4f72-8286-ac6539958bfd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.718357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.720689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:20.815755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.816385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.816663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.816939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.817591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.819639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.831439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:20.839755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c941f66-021c-4993-aace-ac3fea3f307c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.845529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.846751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.847027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.847296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] PATH_INFO: `/v3/projects/2d1f6f4e31a84fefbf0db96db7b73493` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.848489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d1f6f4e31a84fefbf0db96db7b73493)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.849012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1065/2150] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:20 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:20.850623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.853927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.867695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be69ca64-5964-4570-aa80-c566c646eaf4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.874429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:20.875195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:20.875480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:20.875777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:20.876527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:20.878273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:20.879946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:20.894080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1066/2151] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:20 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:20.900559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.913844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d1f6f4e31a84fefbf0db96db7b73493', 'request_id': 'req-14c33114-08e4-4d06-b78c-15ee188e1f11'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.920650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b825f316-053c-4320-a0de-34dbdb8eb42f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:20.922057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d1f6f4e31a84fefbf0db96db7b73493', 'request_id': 'req-14c33114-08e4-4d06-b78c-15ee188e1f11'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.930070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d1f6f4e31a84fefbf0db96db7b73493', 'request_id': 'req-14c33114-08e4-4d06-b78c-15ee188e1f11'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.949342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] The token cache is being invalidate because project 2d1f6f4e31a84fefbf0db96db7b73493 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:20.952660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.952660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:20.959733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1086/2152] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:20 2022] DELETE /identity/v3/projects/2d1f6f4e31a84fefbf0db96db7b73493 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:20.962611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:20.965491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:20.999446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.000240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.000518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.000975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] PATH_INFO: `/v3/projects/4beacfede754467b97886ec624f33ba6/users/854e1a9ada004e39a17331521c0f4638/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.024714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4beacfede754467b97886ec624f33ba6, user_id=854e1a9ada004e39a17331521c0f4638, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.027847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.046884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1067/2153] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:20 2022] PUT /identity/v3/projects/4beacfede754467b97886ec624f33ba6/users/854e1a9ada004e39a17331521c0f4638/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:21.052232 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:21.055330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:21.065214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c33114-08e4-4d06-b78c-15ee188e1f11 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:21.165289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.166574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.167321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.167785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] PATH_INFO: `/v3/users/bfda808a2ccf48559233c7bc4d88a17d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.169252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfda808a2ccf48559233c7bc4d88a17d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.171337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.184158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6d9ab8-9c57-4e9d-8fc3-c8b9aa598c50 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:21.188952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.189560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.189895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.190166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.191331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.192895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.203396 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1068/2154] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:21 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:21.208608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:21.222472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dda0b01-8729-45fe-a4c0-75470d37b582 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:21.230868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b4a271-0d9b-4d3b-8f33-92a76fa78ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.231574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4a271-0d9b-4d3b-8f33-92a76fa78ed5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.231856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4a271-0d9b-4d3b-8f33-92a76fa78ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.232158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b4a271-0d9b-4d3b-8f33-92a76fa78ed5 admin admin] PATH_INFO: `/v3/projects/4beacfede754467b97886ec624f33ba6/users/854e1a9ada004e39a17331521c0f4638/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.234174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4a271-0d9b-4d3b-8f33-92a76fa78ed5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4beacfede754467b97886ec624f33ba6, user_id=854e1a9ada004e39a17331521c0f4638, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.236958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b4a271-0d9b-4d3b-8f33-92a76fa78ed5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.237089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfda808a2ccf48559233c7bc4d88a17d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.237689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfda808a2ccf48559233c7bc4d88a17d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.238336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfda808a2ccf48559233c7bc4d88a17d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.248083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfda808a2ccf48559233c7bc4d88a17d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.248782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1069/2155] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:21 2022] PUT /identity/v3/projects/4beacfede754467b97886ec624f33ba6/users/854e1a9ada004e39a17331521c0f4638/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:21.253443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:21.259015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86021-1984-403b-af72-22b930203561 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.259291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86021-1984-403b-af72-22b930203561 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.259540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce86021-1984-403b-af72-22b930203561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.263314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfda808a2ccf48559233c7bc4d88a17d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.286724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1087/2156] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:20 2022] DELETE /identity/v3/users/bfda808a2ccf48559233c7bc4d88a17d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:21.293903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9ce86021-1984-403b-af72-22b930203561 None None] MFA Rules not processed for user `854e1a9ada004e39a17331521c0f4638`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:21.295965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9ce86021-1984-403b-af72-22b930203561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:21.355300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:21.388256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1070/2157] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:21 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:21.400963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:21.402491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:21.472532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e07ff5-74c5-459f-a21f-e068d89f41ec admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:21.476431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.477052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.477326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.477591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.479341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:21.549045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.550525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.563387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1088/2158] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:38:21.574010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce86021-1984-403b-af72-22b930203561 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:21.576663 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9ce86021-1984-403b-af72-22b930203561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:21.649616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce86021-1984-403b-af72-22b930203561 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:21.654443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.655172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.655520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.655802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] PATH_INFO: `/v3/projects/7a662f4100d344b08152953807e07c6d` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.656951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a662f4100d344b08152953807e07c6d)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.659498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.709935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a662f4100d344b08152953807e07c6d', 'request_id': 'req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.716107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a662f4100d344b08152953807e07c6d', 'request_id': 'req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.724887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a662f4100d344b08152953807e07c6d', 'request_id': 'req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.737488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] The token cache is being invalidate because project 7a662f4100d344b08152953807e07c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:21.737775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.738468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.739965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1071/2159] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:21 2022] DELETE /identity/v3/projects/7a662f4100d344b08152953807e07c6d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:21.744114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:21.755731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb27fc3-07fa-4407-89e3-e4271aad2951 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:21.821274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:21.822033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:21.822416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:21.822766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] PATH_INFO: `/v3/users/f5a1756c468d437e86e60a2ea2c72cf1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:21.824275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5a1756c468d437e86e60a2ea2c72cf1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:21.828892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:21.871764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=7,keystone:SELECT=93 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:21.900756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5a1756c468d437e86e60a2ea2c72cf1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.901480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5a1756c468d437e86e60a2ea2c72cf1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.902372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5a1756c468d437e86e60a2ea2c72cf1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.910491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5a1756c468d437e86e60a2ea2c72cf1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.924117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f169b806-c9cd-438b-ae61-0277c256b28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5a1756c468d437e86e60a2ea2c72cf1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:21.936670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1089/2160] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:21 2022] DELETE /identity/v3/users/f5a1756c468d437e86e60a2ea2c72cf1 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:22.217565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.219150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:22.244271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fee928-c4a7-4f0d-8613-8a9396e4a27b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.309371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.309983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.310355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.310678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] PATH_INFO: `/v3/projects/f73a6547d79645ee88ca08ceb25afea3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.311918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f73a6547d79645ee88ca08ceb25afea3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.313921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.321565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:22.375435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f73a6547d79645ee88ca08ceb25afea3', 'request_id': 'req-dcec045e-c521-4b2b-8adc-66f01672ba20'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:22.381042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f73a6547d79645ee88ca08ceb25afea3', 'request_id': 'req-dcec045e-c521-4b2b-8adc-66f01672ba20'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:22.388693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f73a6547d79645ee88ca08ceb25afea3', 'request_id': 'req-dcec045e-c521-4b2b-8adc-66f01672ba20'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:22.401298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] The token cache is being invalidate because project f73a6547d79645ee88ca08ceb25afea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:22.401632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:22.402662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dcec045e-c521-4b2b-8adc-66f01672ba20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:22.404193 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1072/2161] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:22 2022] DELETE /identity/v3/projects/f73a6547d79645ee88ca08ceb25afea3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:22.409613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.410094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.410245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.430653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:22.431858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:22.490603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1090/2162] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:22.498500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955940b-2987-482e-ba97-b2de8275795b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.498783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955940b-2987-482e-ba97-b2de8275795b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.499056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6955940b-2987-482e-ba97-b2de8275795b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.519163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6955940b-2987-482e-ba97-b2de8275795b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:22.520381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6955940b-2987-482e-ba97-b2de8275795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:22.530124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1073/2163] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:22.534770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.544427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7edce7-d8b1-4db7-bceb-c6238b681811 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.549304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.549949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.550175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.550488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.551136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:22.551439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.553033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.565230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1091/2164] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:22.569686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955940b-2987-482e-ba97-b2de8275795b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.579082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6955940b-2987-482e-ba97-b2de8275795b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.584246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.584846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.585125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.585816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.586003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.587954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.617514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1074/2165] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:22 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:22.621627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.632202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2d5fc3-4ca2-4b78-bf37-cea50540270c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.689684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.690891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.691170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.691436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.692044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.693944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.715376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1092/2166] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:22 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:38:22.719705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.730042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a24ed2-9eda-440b-9da4-ac8845d66e00 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.737375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.737996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.738309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.738580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.739217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.740807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.751010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1075/2167] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:22.755524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.765800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c784b-9eb5-419b-9cef-8a7dec161815 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.771499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a095f4f-cceb-41c0-958f-bd7f0e53bf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.772101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095f4f-cceb-41c0-958f-bd7f0e53bf1c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.772382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095f4f-cceb-41c0-958f-bd7f0e53bf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.772657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a095f4f-cceb-41c0-958f-bd7f0e53bf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.773301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095f4f-cceb-41c0-958f-bd7f0e53bf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.774926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a095f4f-cceb-41c0-958f-bd7f0e53bf1c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.785575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1093/2168] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:22.789570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.801186 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc15c56-d953-41d9-9d3d-19fa996628f4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:22.805962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:22.806674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.806953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.807222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] PATH_INFO: `/v3/projects/97e2f9bd5a7b4fc8828847d518968bfd/users/5439318804a745d98ee897fb511ed9af/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.823828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e2f9bd5a7b4fc8828847d518968bfd, user_id=5439318804a745d98ee897fb511ed9af, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:22.826496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:22.840467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1076/2169] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:22 2022] PUT /identity/v3/projects/97e2f9bd5a7b4fc8828847d518968bfd/users/5439318804a745d98ee897fb511ed9af/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:22.845737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb11f24-067d-4dfc-8a7f-bd3f89a05731 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:22.847560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb11f24-067d-4dfc-8a7f-bd3f89a05731 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:22.847840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb11f24-067d-4dfc-8a7f-bd3f89a05731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:22.867391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-acb11f24-067d-4dfc-8a7f-bd3f89a05731 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:22.868602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-acb11f24-067d-4dfc-8a7f-bd3f89a05731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:22.878861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:22.952960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1094/2170] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:22.960183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:22.961983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:23.018535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293d956d-29cb-425c-8650-51421aef846b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:23.022316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:23.022914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:23.023217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:23.023487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:23.032408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:23.034400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:23.043735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1077/2171] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:23.755743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:23.756270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:23.756350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:23.785025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] MFA Rules not processed for user `5439318804a745d98ee897fb511ed9af`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:23.786321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:23.865815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1095/2172] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:23.876865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:23.879142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:23.918070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:23.946073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f3c7e-8af5-496e-930f-e899c56f6b5a service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:23.949798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:23.950437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:23.950726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:23.951131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:23.961349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:23.962758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:23.972712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1078/2173] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:38:24.196468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:24.198282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:24.214060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae66f52-202d-41b5-a42f-68aff99e9601 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:24.218043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:24.218982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:24.218982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:24.219283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:24.221772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:24.297223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:24.299161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81731667-aca0-4276-ad35-9ae93c27396b service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:24.322886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1096/2174] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:25.560866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:25.569381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b28f0cd-5631-49c0-97b8-87a844a93d4a service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:25.572893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:25.573476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:25.573751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:25.574019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:25.576159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:25.644977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:25.646460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:25.655839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1079/2175] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:25 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:26.556883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb502-59ce-4a07-bc2a-0c7931d5afae None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:26.557466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb502-59ce-4a07-bc2a-0c7931d5afae None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:26.557466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fb502-59ce-4a07-bc2a-0c7931d5afae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:26.576370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7c9fb502-59ce-4a07-bc2a-0c7931d5afae None None] MFA Rules not processed for user `5439318804a745d98ee897fb511ed9af`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:26.577594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7c9fb502-59ce-4a07-bc2a-0c7931d5afae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:26.585628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1097/2176] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:31.049840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:31.051401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:31.064704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e030de-3598-4698-b29c-c7e2a1907729 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:31.069494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:31.070135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:31.070446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:31.070718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] PATH_INFO: `/v3/users/c2f8fa2b7d1545c8ad9543f2c9a25d84` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:31.071866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2f8fa2b7d1545c8ad9543f2c9a25d84)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:31.073849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:31.137430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2f8fa2b7d1545c8ad9543f2c9a25d84'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.138102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2f8fa2b7d1545c8ad9543f2c9a25d84'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.138749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2f8fa2b7d1545c8ad9543f2c9a25d84'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.146598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2f8fa2b7d1545c8ad9543f2c9a25d84'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.154601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2f8fa2b7d1545c8ad9543f2c9a25d84'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.177402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1080/2177] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:31 2022] DELETE /identity/v3/users/c2f8fa2b7d1545c8ad9543f2c9a25d84 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:31.184605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:31.185062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:31.185175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:31.204865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:31.206745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:31.244307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:31.276377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1098/2178] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:31.286610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:31.288055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:31.334347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:31.348055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477ea54d-064e-4e00-b12f-e9ec9e230e6d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:31.351756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:31.352432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:31.352709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:31.352979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:31.362692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:31.364130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:31.374064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1081/2179] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:31.648224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:31.649740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:31.662410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65afc85c-c2e8-4d39-ba8a-b16149628fc5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:31.668658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:31.669303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:31.669584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:31.669867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] PATH_INFO: `/v3/projects/650b85b0583644bfa2bf7ebde89fb701` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:31.671020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] RBAC: Authorizing `identity:delete_project(project_id=650b85b0583644bfa2bf7ebde89fb701)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:31.673252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:31.724352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '650b85b0583644bfa2bf7ebde89fb701', 'request_id': 'req-e7dc58ec-cdec-4758-bf9a-d23315374087'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.729698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '650b85b0583644bfa2bf7ebde89fb701', 'request_id': 'req-e7dc58ec-cdec-4758-bf9a-d23315374087'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.736065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '650b85b0583644bfa2bf7ebde89fb701', 'request_id': 'req-e7dc58ec-cdec-4758-bf9a-d23315374087'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.749841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] The token cache is being invalidate because project 650b85b0583644bfa2bf7ebde89fb701 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:31.750139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.750857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.752377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1099/2180] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:31 2022] DELETE /identity/v3/projects/650b85b0583644bfa2bf7ebde89fb701 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:31.756502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:31.766538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16081f2e-0807-4ed9-bc1e-8631f0d8817b service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:31.834766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:31.835414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:31.835695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:31.835971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] PATH_INFO: `/v3/users/e821c794d7ae4cbd97be7ad4759eabed` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:31.837126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] RBAC: Authorizing `identity:delete_user(user_id=e821c794d7ae4cbd97be7ad4759eabed)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:31.839109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:31.893928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e821c794d7ae4cbd97be7ad4759eabed'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.895162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e821c794d7ae4cbd97be7ad4759eabed'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.895939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e821c794d7ae4cbd97be7ad4759eabed'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.904603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e821c794d7ae4cbd97be7ad4759eabed'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.913781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e821c794d7ae4cbd97be7ad4759eabed'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:31.933255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1082/2181] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:31 2022] DELETE /identity/v3/users/e821c794d7ae4cbd97be7ad4759eabed => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:32.230242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:32.235196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:32.257982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc58ec-cdec-4758-bf9a-d23315374087 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:32.333607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:32.334284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:32.334567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:32.334842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] PATH_INFO: `/v3/projects/fb21d5e8df3c4a7fa25d092848e7cca8` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:32.335978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb21d5e8df3c4a7fa25d092848e7cca8)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:32.337933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:32.385738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb21d5e8df3c4a7fa25d092848e7cca8', 'request_id': 'req-5cb1770a-7124-4734-af5c-f66ae7175266'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.391293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb21d5e8df3c4a7fa25d092848e7cca8', 'request_id': 'req-5cb1770a-7124-4734-af5c-f66ae7175266'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.397592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb21d5e8df3c4a7fa25d092848e7cca8', 'request_id': 'req-5cb1770a-7124-4734-af5c-f66ae7175266'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.399280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:32.412049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] The token cache is being invalidate because project fb21d5e8df3c4a7fa25d092848e7cca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:32.412416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.413412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.415346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1100/2182] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:32 2022] DELETE /identity/v3/projects/fb21d5e8df3c4a7fa25d092848e7cca8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:32.419557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:32.421065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:32.434961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb77be0-a853-4a4e-af4a-1bbb856bc4be admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:32.509104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:32.509733 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:32.510006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:32.510309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] PATH_INFO: `/v3/users/854e1a9ada004e39a17331521c0f4638` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:32.511474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] RBAC: Authorizing `identity:delete_user(user_id=854e1a9ada004e39a17331521c0f4638)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:32.513414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:32.518225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:32.582538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854e1a9ada004e39a17331521c0f4638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.583268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854e1a9ada004e39a17331521c0f4638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.583955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854e1a9ada004e39a17331521c0f4638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.592467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854e1a9ada004e39a17331521c0f4638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.601599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-c5d7cb0c-09fd-4a3b-83c7-dabc2b5e024d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854e1a9ada004e39a17331521c0f4638'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:32.622276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1083/2183] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:32 2022] DELETE /identity/v3/users/854e1a9ada004e39a17331521c0f4638 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:33.062933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.064777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:33.079673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb1770a-7124-4734-af5c-f66ae7175266 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.199107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.199792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.200078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.200604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] PATH_INFO: `/v3/projects/4beacfede754467b97886ec624f33ba6` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.202012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4beacfede754467b97886ec624f33ba6)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.204076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.285525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4beacfede754467b97886ec624f33ba6', 'request_id': 'req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:33.297867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4beacfede754467b97886ec624f33ba6', 'request_id': 'req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:33.322842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4beacfede754467b97886ec624f33ba6', 'request_id': 'req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:33.353368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] The token cache is being invalidate because project 4beacfede754467b97886ec624f33ba6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:33.353485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:33.354213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f024d43d-5b6d-4c2d-88c6-7bef31aafd73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:33.356253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1101/2184] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:33 2022] DELETE /identity/v3/projects/4beacfede754467b97886ec624f33ba6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:33.361167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.361527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.361813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.390918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:33.392420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:33.458365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1084/2185] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:33.466257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.466628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.466967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.488394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:33.489719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:33.497662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1102/2186] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:33.502840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.513369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa62799-7600-42d3-befa-8b70197ab768 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.519037 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.519656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.519931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.520203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.520863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:33.521175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.522852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.535191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1085/2187] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:33.540421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.550717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1157b-66f8-40ed-b7f4-1a32f988b009 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.562546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.563103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.563376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.563642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.565154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.567099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.594763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1103/2188] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:33 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:33.599890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.610121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7872c0c7-94dd-4628-ac50-ae19ee06dbba admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.671645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.672256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.672815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.673085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.673784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.675879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.693474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:33.698777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1086/2189] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:33 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:33.703104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.715129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b81375-bad4-4245-9388-0218ffec9a6b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.720902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.721876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.722330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.722687 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.723510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.725124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.739756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1104/2190] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:33.743965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.756151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d26319f-3050-4de1-baf7-67e136852632 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.760942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f39678b-fd72-4f67-9f68-09e5fa52515b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.761546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39678b-fd72-4f67-9f68-09e5fa52515b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.761848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39678b-fd72-4f67-9f68-09e5fa52515b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.762118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f39678b-fd72-4f67-9f68-09e5fa52515b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.762795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39678b-fd72-4f67-9f68-09e5fa52515b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.764438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f39678b-fd72-4f67-9f68-09e5fa52515b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.776744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1087/2191] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:33.781683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.792676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb12d2cf-893b-4a24-a43e-3113d88d099d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:33.797336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:33.798285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.798558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.799368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] PATH_INFO: `/v3/projects/cc4ed4d70a074b97acf12130e4b00bcb/users/c0c89aa8c55042e7af5452f1a94318a9/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.815693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc4ed4d70a074b97acf12130e4b00bcb, user_id=c0c89aa8c55042e7af5452f1a94318a9, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:33.818468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:33.831894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1105/2192] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:33 2022] PUT /identity/v3/projects/cc4ed4d70a074b97acf12130e4b00bcb/users/c0c89aa8c55042e7af5452f1a94318a9/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:33.838628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:33.838628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:33.838628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:33.860678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:33.862419 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:33.932176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1088/2193] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:33.942753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:33.944509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:33.964889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=4,keystone:DELETE=8,keystone:SELECT=88 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:34.004901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6ecec-02f6-4288-9f2b-c38527ad8b97 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:34.008409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d02153-eb8d-4338-b037-92651380c6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:34.009011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d02153-eb8d-4338-b037-92651380c6aa service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:34.009287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d02153-eb8d-4338-b037-92651380c6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:34.009553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d02153-eb8d-4338-b037-92651380c6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:34.019532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d02153-eb8d-4338-b037-92651380c6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:34.020957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d02153-eb8d-4338-b037-92651380c6aa service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:34.030700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1106/2194] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:38.557361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:38.558986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:38.572309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf37918-2f97-48a5-b3a5-d926daf3ec56 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:38.577451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:38.578083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:38.578506 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:38.578787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] PATH_INFO: `/v3/users/7f13c3c09c2b48d58924139701a85084` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:38.580645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f13c3c09c2b48d58924139701a85084)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:38.582649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:38.699229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f13c3c09c2b48d58924139701a85084'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:38.700406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f13c3c09c2b48d58924139701a85084'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:38.701703 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f13c3c09c2b48d58924139701a85084'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:38.724578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f13c3c09c2b48d58924139701a85084'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:38.741979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f13c3c09c2b48d58924139701a85084'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:38.799743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1089/2195] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:38 2022] DELETE /identity/v3/users/7f13c3c09c2b48d58924139701a85084 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:38.801857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:38.802388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:38.802575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:38.838276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:38.840819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:38.998336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1107/2196] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:39.006163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:39.007888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.089463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54569be5-4d66-4b54-949f-25f53807c98c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:39.099471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7d539-45b6-4e35-80cf-9c5e94e57c50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:39.099471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d539-45b6-4e35-80cf-9c5e94e57c50 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.099471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d539-45b6-4e35-80cf-9c5e94e57c50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.099932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7d539-45b6-4e35-80cf-9c5e94e57c50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.117000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7d539-45b6-4e35-80cf-9c5e94e57c50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:39.119411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7d539-45b6-4e35-80cf-9c5e94e57c50 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:39.145406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1090/2197] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:39.611817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:39.613459 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.627908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685c9092-44c8-4766-93aa-5481f5b7cdc6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:39.640468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:39.644599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.645945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.646358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] PATH_INFO: `/v3/projects/6d147de1407f425e91ee57a4be743179` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.658544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d147de1407f425e91ee57a4be743179)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:39.661979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:39.743935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d147de1407f425e91ee57a4be743179', 'request_id': 'req-6212fe79-7202-43fb-94ad-548421eff169'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:39.755611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d147de1407f425e91ee57a4be743179', 'request_id': 'req-6212fe79-7202-43fb-94ad-548421eff169'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:39.766063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d147de1407f425e91ee57a4be743179', 'request_id': 'req-6212fe79-7202-43fb-94ad-548421eff169'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:39.790653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.791608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.792380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] The token cache is being invalidate because project 6d147de1407f425e91ee57a4be743179 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:39.792684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:39.793094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.793420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6212fe79-7202-43fb-94ad-548421eff169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:39.799838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1108/2198] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:39 2022] DELETE /identity/v3/projects/6d147de1407f425e91ee57a4be743179 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:39.803293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:39.803462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d3128-abf4-48cb-9cdc-dbf52a6e9cae None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.803921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d3128-abf4-48cb-9cdc-dbf52a6e9cae None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.804008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d3128-abf4-48cb-9cdc-dbf52a6e9cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.837036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1b0d3128-abf4-48cb-9cdc-dbf52a6e9cae None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:39.838307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1b0d3128-abf4-48cb-9cdc-dbf52a6e9cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.850016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] MFA Rules not processed for user `c0c89aa8c55042e7af5452f1a94318a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:39.853405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.902458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:39.937028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1109/2199] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:39.945815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.946100 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.946401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.957330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1091/2200] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:39.961614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:39.965960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.981893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:39.983237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:39.985777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fa7ed0-1be8-417b-ab25-231b9334b8c0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:39.990709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:39.991299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:39.991592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:39.991860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:39.992503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:39.996661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1110/2201] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:39.996816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:39.998529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.010155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd16e907-9409-493b-94d2-580b1c9c05b8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.015530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.016305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.016588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.016857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.018123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:40.018466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.020077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.037961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1111/2202] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:38:40.044978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1092/2203] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:39 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:40.044978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.055228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.068084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eea880-5aa8-4350-8698-85f1bc8aa731 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.133674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99165aa9-6d22-4ea2-921b-4eecbab59ec5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.139053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.139757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.140136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.140529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.141282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.143474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.150019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.150633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.151075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.151359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.151996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.154099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.185929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1093/2204] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:40 2022] POST /identity/v3/projects => generated 373 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:40.188254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1112/2205] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:40 2022] POST /identity/v3/users => generated 291 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:40.188400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.195622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.207427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cf183-485d-4730-8cf9-d624a505d034 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.270255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:40.275673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec9056-eb87-4724-ada0-73cae72f380e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.276135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.276752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.277025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.277293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.277943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.279918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.283793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.284412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.285337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.285627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.286384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.288342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.294008 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1113/2206] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:40.298220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.308821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77072a86-4cdd-4ea1-b033-9e540a674a5a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.310968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1094/2207] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:40 2022] POST /identity/v3/users => generated 287 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:40.315548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.318067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.318776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.319064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.319345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] PATH_INFO: `/v3/projects/318246d199744f0681068cf0310d0f67/users/bdf8485a95454f7d80eb4b31656006fd/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.325822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6483faf-c718-442e-8343-faaa319a0e0c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.331802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.332408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.332689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.332960 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.333678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.335431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.336379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] RBAC: Authorizing `identity:create_grant(project_id=318246d199744f0681068cf0310d0f67, user_id=bdf8485a95454f7d80eb4b31656006fd, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.339128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.347893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1095/2208] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:40.351826 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.353195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1114/2209] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:40 2022] PUT /identity/v3/projects/318246d199744f0681068cf0310d0f67/users/bdf8485a95454f7d80eb4b31656006fd/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:40.356829 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.363618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2742f9e-b7bf-4dfc-87da-c36283b1e53c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.393965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:40.428361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.429045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.429324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.429601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.430275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.431896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.434482 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79568c-c0a7-441e-9369-50b393879b33 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.443268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.443912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1096/2210] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:40.446986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.447583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.448491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.449156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.449976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.452236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.460526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d350a-319f-48cd-a7c0-94695023fe03 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.465377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1115/2211] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:40 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:40.465627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edb893e-a133-411d-9754-ed2fa7d8fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.466380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb893e-a133-411d-9754-ed2fa7d8fc78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.466660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb893e-a133-411d-9754-ed2fa7d8fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.467158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edb893e-a133-411d-9754-ed2fa7d8fc78 admin admin] PATH_INFO: `/v3/projects/a99d71b4272f4331af80a15a913bca00/users/93d92f5e1d5f491b949399f6bce69a33/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.469606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.482809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5af04b-b81d-4050-8307-142c83045e54 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.485300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb893e-a133-411d-9754-ed2fa7d8fc78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a99d71b4272f4331af80a15a913bca00, user_id=93d92f5e1d5f491b949399f6bce69a33, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.488647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edb893e-a133-411d-9754-ed2fa7d8fc78 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.489068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.489771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.490052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.490457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] PATH_INFO: `/v3/projects/318246d199744f0681068cf0310d0f67/users/bdf8485a95454f7d80eb4b31656006fd/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.492514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] RBAC: Authorizing `identity:create_grant(project_id=318246d199744f0681068cf0310d0f67, user_id=bdf8485a95454f7d80eb4b31656006fd, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.495223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.514253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1097/2212] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:40 2022] PUT /identity/v3/projects/a99d71b4272f4331af80a15a913bca00/users/93d92f5e1d5f491b949399f6bce69a33/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:40.521506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1116/2213] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:40 2022] PUT /identity/v3/projects/318246d199744f0681068cf0310d0f67/users/bdf8485a95454f7d80eb4b31656006fd/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:40.524793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.525161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.525560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.560397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] MFA Rules not processed for user `93d92f5e1d5f491b949399f6bce69a33`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:40.561979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:40.730646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1098/2214] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:40 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:40.745293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:40.747253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:40.875474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7bb60-5575-47e4-84bb-b40d2dd2399a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:40.880674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41ebd2b-0844-4c11-95e2-57891578178e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:40.881202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ebd2b-0844-4c11-95e2-57891578178e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:40.881476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ebd2b-0844-4c11-95e2-57891578178e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:40.881749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41ebd2b-0844-4c11-95e2-57891578178e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:40.893601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ebd2b-0844-4c11-95e2-57891578178e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:40.895218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41ebd2b-0844-4c11-95e2-57891578178e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:40.908377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1117/2215] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:44.673874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:44.792699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:44.871100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbd74f-ac60-43d0-85d6-8c776fb4d660 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:44.876373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:44.877005 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:44.877285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:44.877563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] PATH_INFO: `/v3/users/93d92f5e1d5f491b949399f6bce69a33` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:44.878843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93d92f5e1d5f491b949399f6bce69a33)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:44.880900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:44.981896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93d92f5e1d5f491b949399f6bce69a33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:44.982874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93d92f5e1d5f491b949399f6bce69a33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:44.987166 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93d92f5e1d5f491b949399f6bce69a33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:44.996947 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93d92f5e1d5f491b949399f6bce69a33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:45.007903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93d92f5e1d5f491b949399f6bce69a33'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:45.037999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1099/2216] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:44 2022] DELETE /identity/v3/users/93d92f5e1d5f491b949399f6bce69a33 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:45.042977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:45.043542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:45.043542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:45.090898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:45.092055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:45.214899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1118/2217] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:45.225922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:45.228536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:45.372080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa8cd30-17cb-45cd-bfc8-39a88e1cf2f0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:45.380029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc24294-b7b7-42c1-aac3-2740086e0ed7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:45.382131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc24294-b7b7-42c1-aac3-2740086e0ed7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:45.382515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc24294-b7b7-42c1-aac3-2740086e0ed7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:45.383456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc24294-b7b7-42c1-aac3-2740086e0ed7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:45.410834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc24294-b7b7-42c1-aac3-2740086e0ed7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:45.412336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc24294-b7b7-42c1-aac3-2740086e0ed7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:45.442735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1100/2218] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:46.389236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:46.391262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:46.416375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4bfc4-9ce4-4816-9063-3ac3a92cc67b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:46.422899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:46.424658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:46.425129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:46.425578 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] PATH_INFO: `/v3/projects/a99d71b4272f4331af80a15a913bca00` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:46.427118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a99d71b4272f4331af80a15a913bca00)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:46.429907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:46.509026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:46.512299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a99d71b4272f4331af80a15a913bca00', 'request_id': 'req-31f7cbd5-c069-4916-8861-fc19d994876e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:46.522316 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a99d71b4272f4331af80a15a913bca00', 'request_id': 'req-31f7cbd5-c069-4916-8861-fc19d994876e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:46.532174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a99d71b4272f4331af80a15a913bca00', 'request_id': 'req-31f7cbd5-c069-4916-8861-fc19d994876e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:46.551281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] The token cache is being invalidate because project a99d71b4272f4331af80a15a913bca00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:46.551583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:46.552542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-31f7cbd5-c069-4916-8861-fc19d994876e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:46.554960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1119/2219] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:46 2022] DELETE /identity/v3/projects/a99d71b4272f4331af80a15a913bca00 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:46.560769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:46.561280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:46.561371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:46.584289 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:46.585562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:46.659755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:46.670921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1101/2220] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:46.679066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:46.679438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:46.680741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:46.706882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:46.708943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:46.722110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1120/2221] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:46.726158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:46.752954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac08036-0c35-41ed-bc55-0798ddc6cb96 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:46.765458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:46.767073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:46.768644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:46.772784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:46.773835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:46.774386 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:46.776881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:46.816023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1102/2222] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:46.820552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:46.833738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39087561-3c38-4900-8d92-566fdbd748cc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:46.844919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:46.846837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:46.847232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:46.847701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:46.848472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:46.853339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:46.919805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1121/2223] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:46 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:46.925629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:46.937578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce480ef1-06ed-4f38-b42b-e30b91036552 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:47.077910 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:47.079107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:47.080530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:47.081018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:47.082133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:47.096033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:47.139169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1103/2224] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:46 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:47.144636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:47.159246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813228b8-6ce1-472c-9e6a-5d3ea0a3de67 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:47.164473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:47.165191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:47.165789 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:47.166232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:47.167118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:47.168977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:47.188820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:47.201737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1122/2225] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:47.203569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5364c606-af79-4fb7-8b4f-5a0f3942630e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:47.209721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7b44b9-ad3f-40e5-ab26-27d51f87966f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:47.210530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b44b9-ad3f-40e5-ab26-27d51f87966f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:47.210879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b44b9-ad3f-40e5-ab26-27d51f87966f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:47.211223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7b44b9-ad3f-40e5-ab26-27d51f87966f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:47.212002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b44b9-ad3f-40e5-ab26-27d51f87966f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:47.213837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7b44b9-ad3f-40e5-ab26-27d51f87966f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:47.254668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:47.254878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1104/2226] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:47 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:47.276022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243d5c3-6cfd-4a85-8c24-d84aacec32d8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:47.281154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:47.281827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:47.282110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:47.282437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] PATH_INFO: `/v3/projects/76c2e8ca4e594275a764e2bfd287af9d/users/ab3dbf21871d4ee284f3c155a1b70ce7/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:47.300504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76c2e8ca4e594275a764e2bfd287af9d, user_id=ab3dbf21871d4ee284f3c155a1b70ce7, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:47.305278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:47.322574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1123/2227] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:47 2022] PUT /identity/v3/projects/76c2e8ca4e594275a764e2bfd287af9d/users/ab3dbf21871d4ee284f3c155a1b70ce7/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:47.333275 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e4fd-3bd4-4027-8a5b-0e106112893f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:47.333594 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e4fd-3bd4-4027-8a5b-0e106112893f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:47.333830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c9e4fd-3bd4-4027-8a5b-0e106112893f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:47.367772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-99c9e4fd-3bd4-4027-8a5b-0e106112893f None None] MFA Rules not processed for user `ab3dbf21871d4ee284f3c155a1b70ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:47.369301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-99c9e4fd-3bd4-4027-8a5b-0e106112893f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:47.456346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1105/2228] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:47.467548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:47.470130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:47.538172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3982c2-08a0-43aa-bd42-45c2bc209da9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:47.541897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:47.542554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:47.543439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:47.543731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:47.554060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:47.555849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:47.586577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1124/2229] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:49.305847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20a483-b303-41f4-92b4-df290e623818 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:49.305847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20a483-b303-41f4-92b4-df290e623818 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:49.307089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20a483-b303-41f4-92b4-df290e623818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:49.344550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6d20a483-b303-41f4-92b4-df290e623818 None None] MFA Rules not processed for user `bdf8485a95454f7d80eb4b31656006fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:49.346665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6d20a483-b303-41f4-92b4-df290e623818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:49.416247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:49.462395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1106/2230] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:49 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:49.472686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:49.474354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:49.611988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da4525c-8d0f-427f-a179-b36c7fa00564 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:49.619083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:49.619717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:49.619998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:49.620269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:49.623555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:49.645371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:49.770685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:49.772336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec2301-970a-4e45-bf20-d7c0cdd2555e service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:49.792777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1125/2231] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:49.941610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690b2f2-1035-46a6-af67-b5ae6b989ef7 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:49.942411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690b2f2-1035-46a6-af67-b5ae6b989ef7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:49.945897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f690b2f2-1035-46a6-af67-b5ae6b989ef7 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:49.948948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1107/2232] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:38:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:38:49.971157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:49.971439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:49.971716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:50.011375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] MFA Rules not processed for user `43c98ae03bf9469bb70ae0ee2e9aecb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:50.015863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:50.083867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690b2f2-1035-46a6-af67-b5ae6b989ef7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:50.086476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f690b2f2-1035-46a6-af67-b5ae6b989ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:50.138024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1126/2233] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:38:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:50.153454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:50.155567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:50.173357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3ef8e6-eb96-45d0-902a-ebae3491e8b6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:50.182855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e8b2c-0404-4d40-aaad-73bfd075fa08 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:50.183565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e8b2c-0404-4d40-aaad-73bfd075fa08 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:50.183855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e8b2c-0404-4d40-aaad-73bfd075fa08 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:50.184300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e8b2c-0404-4d40-aaad-73bfd075fa08 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:50.197276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e8b2c-0404-4d40-aaad-73bfd075fa08 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:50.210225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e8b2c-0404-4d40-aaad-73bfd075fa08 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:50.216897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1127/2234] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:50.227755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:50.228066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:50.229824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:50.272447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] MFA Rules not processed for user `ab3dbf21871d4ee284f3c155a1b70ce7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:50.273625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f690b2f2-1035-46a6-af67-b5ae6b989ef7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:50.274117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:50.280373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:50.281348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:50.281849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:50.282353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] PATH_INFO: `/v3/users/7f1047b63c4148ca8073e3414ef84d28` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:50.284786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1128/2235] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:50.285948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f1047b63c4148ca8073e3414ef84d28)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:50.288345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:50.399556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f1047b63c4148ca8073e3414ef84d28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.401809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f1047b63c4148ca8073e3414ef84d28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.403065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f1047b63c4148ca8073e3414ef84d28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.417494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f1047b63c4148ca8073e3414ef84d28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.429041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ecf3d7a9-96e2-4b48-8b7b-1667437bab4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f1047b63c4148ca8073e3414ef84d28'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.455690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1108/2236] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:50 2022] DELETE /identity/v3/users/7f1047b63c4148ca8073e3414ef84d28 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:38:50.703180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:50.727619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca95bb1-4921-4ef6-9010-1c8a275eb210 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:50.847956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:50.849155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:50.850016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:50.850420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] PATH_INFO: `/v3/projects/53aa6850ae4d44619cc27cef9dcadc61` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:50.852078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53aa6850ae4d44619cc27cef9dcadc61)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:50.854437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:50.902983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:50.927771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53aa6850ae4d44619cc27cef9dcadc61', 'request_id': 'req-bf945241-1a83-455e-a983-1a166a4088c3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.943566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53aa6850ae4d44619cc27cef9dcadc61', 'request_id': 'req-bf945241-1a83-455e-a983-1a166a4088c3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.956336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53aa6850ae4d44619cc27cef9dcadc61', 'request_id': 'req-bf945241-1a83-455e-a983-1a166a4088c3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.978420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] The token cache is being invalidate because project 53aa6850ae4d44619cc27cef9dcadc61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:50.979338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.979740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-bf945241-1a83-455e-a983-1a166a4088c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:50.982287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1129/2237] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:50 2022] DELETE /identity/v3/projects/53aa6850ae4d44619cc27cef9dcadc61 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:50.987515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:50.987802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:50.988080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.024108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:51.025352 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:51.132527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:51.154841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1109/2238] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:51.162702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.165189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.165670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.203920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:51.205671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:51.221994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1130/2239] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:51.227891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:51.241152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf045225-29d3-4ce1-a9c9-311e79b95fd0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:51.251244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:51.255292 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.255712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.256121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.256894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:51.257692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:51.259513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:51.296425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1110/2240] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:51.301924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:51.321728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91f779f-fd44-4b61-8571-98e401ccc517 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:51.333680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:51.334648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.338855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.339361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.341563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:51.345894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:51.425663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1131/2241] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:51 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:51.429735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:51.458567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64262873-4466-4338-8b5e-8005c6fe578c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:51.589288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:51.589912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.590229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.590499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.591362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:51.593651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:51.632590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1111/2242] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:51 2022] POST /identity/v3/users => generated 287 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:51.638383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:51.658063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8969769-039b-4652-96d1-18f8556aef4a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:51.663157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:51.663802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.664082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.664353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.665589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:51.666651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:51.683245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:51.684349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1132/2243] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:51.696849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acb9af8-8482-4244-99af-eb06abb19d58 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:51.703636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab336409-b10a-4f95-8dc3-b65faa760916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:51.704274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab336409-b10a-4f95-8dc3-b65faa760916 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.704553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab336409-b10a-4f95-8dc3-b65faa760916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.705455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab336409-b10a-4f95-8dc3-b65faa760916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.706177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab336409-b10a-4f95-8dc3-b65faa760916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:51.708417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab336409-b10a-4f95-8dc3-b65faa760916 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:51.726872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1112/2244] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:51.729448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:51.745006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9137e4c-b3d1-4e4c-b8ab-406ef1f67c44 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:51.752220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:51.752946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.753229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.753502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] PATH_INFO: `/v3/projects/e1243b68f22d47dca0ee1ece2aa1c9c6/users/5211875fe54147789e30dc6972576c1d/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.791743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1243b68f22d47dca0ee1ece2aa1c9c6, user_id=5211875fe54147789e30dc6972576c1d, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:51.796584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:51.826546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1133/2245] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:51 2022] PUT /identity/v3/projects/e1243b68f22d47dca0ee1ece2aa1c9c6/users/5211875fe54147789e30dc6972576c1d/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:51.839925 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618523-2c15-4717-b018-14ffe16628e6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:51.840628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618523-2c15-4717-b018-14ffe16628e6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:51.841138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3618523-2c15-4717-b018-14ffe16628e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:51.890467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f3618523-2c15-4717-b018-14ffe16628e6 None None] MFA Rules not processed for user `5211875fe54147789e30dc6972576c1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:51.891853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f3618523-2c15-4717-b018-14ffe16628e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:51.999943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1113/2246] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:52.013102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:52.017543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:52.162546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca94a8a-d98d-49dc-b560-5a0421511d04 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:52.170154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:52.171045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:52.171414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:52.171888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:52.187056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:52.188652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:52.200465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1134/2247] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:52.225548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60201df4-5020-486e-a518-2d3435f8b10d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:52.225838 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60201df4-5020-486e-a518-2d3435f8b10d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:52.226335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60201df4-5020-486e-a518-2d3435f8b10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:52.248407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-60201df4-5020-486e-a518-2d3435f8b10d None None] MFA Rules not processed for user `c0c89aa8c55042e7af5452f1a94318a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:52.249711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-60201df4-5020-486e-a518-2d3435f8b10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:52.329762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:52.336895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1114/2248] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:55.062444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:55.113209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:55.137753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997dda4d-6a01-4a1e-88a6-def6b54a05e2 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:55.144046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:55.144741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:55.145066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:55.145626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] PATH_INFO: `/v3/users/5211875fe54147789e30dc6972576c1d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:55.146863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5211875fe54147789e30dc6972576c1d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:55.152366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:55.207475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5211875fe54147789e30dc6972576c1d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.208150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5211875fe54147789e30dc6972576c1d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.208786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5211875fe54147789e30dc6972576c1d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.218847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5211875fe54147789e30dc6972576c1d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.235846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5211875fe54147789e30dc6972576c1d'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.255245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1135/2249] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:38:55 2022] DELETE /identity/v3/users/5211875fe54147789e30dc6972576c1d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:55.257625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:55.258521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:55.259268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:55.279546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:55.280795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:55.377779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1115/2250] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:55.389356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:55.391431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:55.472836 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8cdb0e-e936-4d37-8bf7-d7483e5819f1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:55.477679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a98b47-690f-4483-9e48-d63235e5dab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:55.478454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a98b47-690f-4483-9e48-d63235e5dab1 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:55.478805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a98b47-690f-4483-9e48-d63235e5dab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:55.479150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a98b47-690f-4483-9e48-d63235e5dab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:55.493252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a98b47-690f-4483-9e48-d63235e5dab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:55.494961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a98b47-690f-4483-9e48-d63235e5dab1 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:55.509375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1136/2251] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:55.838332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:55.839869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:55.855638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc8a96-6910-4087-b34c-fc5cf3f7b22c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:55.862721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:55.863448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:55.863834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:55.864342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] PATH_INFO: `/v3/projects/e1243b68f22d47dca0ee1ece2aa1c9c6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:55.865612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1243b68f22d47dca0ee1ece2aa1c9c6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:55.867689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:55.943563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1243b68f22d47dca0ee1ece2aa1c9c6', 'request_id': 'req-cfa61f0d-0434-4cac-9eb5-7165749fb132'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.953261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1243b68f22d47dca0ee1ece2aa1c9c6', 'request_id': 'req-cfa61f0d-0434-4cac-9eb5-7165749fb132'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.964426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1243b68f22d47dca0ee1ece2aa1c9c6', 'request_id': 'req-cfa61f0d-0434-4cac-9eb5-7165749fb132'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.985309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] The token cache is being invalidate because project e1243b68f22d47dca0ee1ece2aa1c9c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:38:55.985309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.985776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-cfa61f0d-0434-4cac-9eb5-7165749fb132 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:38:55.987388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1116/2252] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:38:55 2022] DELETE /identity/v3/projects/e1243b68f22d47dca0ee1ece2aa1c9c6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:55.992997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:55.993370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:55.993728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.015738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:56.017198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:56.163188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1137/2253] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:56.172423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.172705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.173245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.204051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:56.205964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:56.214577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1117/2254] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:56.218928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.223857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:56.229437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d28846b-915e-4346-a2ed-2df3a9c04886 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.236891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.237625 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.237987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.238344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.239166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:38:56.239654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.242095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.258215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1138/2255] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:38:56.260228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.272817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cd61a1-0f24-4399-a283-2b5d6e0db9cd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.279368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d7b9a-e98f-4f8d-9d50-90d5ea1aef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.280055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7b9a-e98f-4f8d-9d50-90d5ea1aef9e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.280404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7b9a-e98f-4f8d-9d50-90d5ea1aef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.280680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d7b9a-e98f-4f8d-9d50-90d5ea1aef9e admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.281319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d7b9a-e98f-4f8d-9d50-90d5ea1aef9e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.284089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689d7b9a-e98f-4f8d-9d50-90d5ea1aef9e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.313316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1118/2256] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:56.317617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.329040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b076f-bbb9-4d4e-8fbd-fe1a17a58acb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.403408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.403408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.403408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.403408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.409295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.409295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.419657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.419946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.420221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.440548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] MFA Rules not processed for user `5439318804a745d98ee897fb511ed9af`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:56.441797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:56.448586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1139/2257] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/users => generated 287 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:56.450103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.461917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db65a38-c522-49d8-ad74-c132308eb47f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.466761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.467428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.467713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.467984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.468621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.471867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.486466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1140/2258] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:56.486841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.497605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d71eb0-17da-4979-83ac-16be25a868b8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.502410 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:56.502963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdcac4c-c896-4234-9db2-79221cd88e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.503695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcac4c-c896-4234-9db2-79221cd88e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.504329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcac4c-c896-4234-9db2-79221cd88e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.504444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcac4c-c896-4234-9db2-79221cd88e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.505079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcac4c-c896-4234-9db2-79221cd88e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.506956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdcac4c-c896-4234-9db2-79221cd88e7b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.517584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1119/2259] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:56.520908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1141/2260] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:38:56.527656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.541062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1166cbcf-1f9a-420e-8dc3-d799dc76567f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.558690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.560684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.561280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.562167 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] PATH_INFO: `/v3/projects/5d72f8ab73384482a4a2216939836da2/users/8c99c4bb79d84331964edba016d2a0c4/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.584079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d72f8ab73384482a4a2216939836da2, user_id=8c99c4bb79d84331964edba016d2a0c4, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.593203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.614911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1120/2261] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:56 2022] PUT /identity/v3/projects/5d72f8ab73384482a4a2216939836da2/users/8c99c4bb79d84331964edba016d2a0c4/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:56.625130 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.625481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.625937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.658198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] MFA Rules not processed for user `8c99c4bb79d84331964edba016d2a0c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:56.659548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:56.726972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1142/2262] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:38:56.731179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.808581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20df33-d013-4d3f-8e8e-da2c32e33637 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.815501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.816249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.816529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.816892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.817582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.819618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.848646 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1121/2263] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:56.853113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.864617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52640eeb-9457-4740-8982-3b14b40b7196 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.910710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:56.925304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.925927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.926230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.926510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.927155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.929071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.955928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1143/2264] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:38:56 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:38:56.957224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:56.967924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2853f1b-7536-4fe2-8bed-b4098a6fd941 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:56.973233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:56.973846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:56.974119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:56.974434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:56.975082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:56.976698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:56.988385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1122/2265] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:56.992669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:57.003772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5becf505-a3fe-4256-907b-3a379d485dfc admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:57.009076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:57.009885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:57.010302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:57.010592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] PATH_INFO: `/v3/projects/94f17d978cee4805a776a222fe28f30d/users/87374d6cb4da416bae5da67d0ae2edfa/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:57.036652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f17d978cee4805a776a222fe28f30d, user_id=87374d6cb4da416bae5da67d0ae2edfa, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:57.039381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:57.055561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1144/2266] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:56 2022] PUT /identity/v3/projects/94f17d978cee4805a776a222fe28f30d/users/87374d6cb4da416bae5da67d0ae2edfa/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:57.059853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:57.073176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d5c0e8-8468-4cd4-8ab6-5a12f26268f8 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:57.141387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dacd9e-6fa2-4ad0-bf50-97e5ba7f4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:57.142779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dacd9e-6fa2-4ad0-bf50-97e5ba7f4a9d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:57.143069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dacd9e-6fa2-4ad0-bf50-97e5ba7f4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:57.143344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dacd9e-6fa2-4ad0-bf50-97e5ba7f4a9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:57.143988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dacd9e-6fa2-4ad0-bf50-97e5ba7f4a9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:57.146365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dacd9e-6fa2-4ad0-bf50-97e5ba7f4a9d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:57.161598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1123/2267] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:38:57 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:38:57.165521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:57.182556 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d583ed4c-16ce-41c0-b92a-d9d6409abf29 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:57.188952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:57.190675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:57.191041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:57.191486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] PATH_INFO: `/v3/projects/94f17d978cee4805a776a222fe28f30d/users/87374d6cb4da416bae5da67d0ae2edfa/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:57.193843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94f17d978cee4805a776a222fe28f30d, user_id=87374d6cb4da416bae5da67d0ae2edfa, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:57.197044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:57.235830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1145/2268] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:38:57 2022] PUT /identity/v3/projects/94f17d978cee4805a776a222fe28f30d/users/87374d6cb4da416bae5da67d0ae2edfa/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:38:57.237285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:57.237597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:57.237911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:57.286649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] MFA Rules not processed for user `87374d6cb4da416bae5da67d0ae2edfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:38:57.287892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:57.334851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:38:57.377630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1124/2269] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:38:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:38:57.389369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:57.391895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:38:57.517735 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6188f94f-9ea8-4236-80c9-2858a45d41e3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:57.523943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:57.524873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:57.525672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:57.526210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:57.545458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:57.545458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:57.549884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1146/2270] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:57 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:38:57.644622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:38:57.655619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9da67bd-2e06-45c2-96bd-06841903b021 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:38:57.659501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287b909c-7a77-4189-bd74-654a3179b855 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:38:57.660185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b909c-7a77-4189-bd74-654a3179b855 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:38:57.660550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b909c-7a77-4189-bd74-654a3179b855 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:38:57.660988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287b909c-7a77-4189-bd74-654a3179b855 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:38:57.727363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b909c-7a77-4189-bd74-654a3179b855 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:38:57.728844 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287b909c-7a77-4189-bd74-654a3179b855 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:38:57.739465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1125/2271] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:38:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:39:07.543892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=49,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:07.733931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=41 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:28.100560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:28.103009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:28.224909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8528e11-91e0-4f53-86e2-5677f3478631 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:28.230749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:28.231481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:28.232014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:28.232637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:28.234673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:28.256912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:28.259249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:28.274611 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1147/2272] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:39:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:39:29.229974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f001a-23d4-488c-8c05-3f2f9074cccb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:29.230581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f001a-23d4-488c-8c05-3f2f9074cccb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:29.230581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f001a-23d4-488c-8c05-3f2f9074cccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:29.255759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-541f001a-23d4-488c-8c05-3f2f9074cccb None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:29.256983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-541f001a-23d4-488c-8c05-3f2f9074cccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:29.316262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1126/2273] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:39:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:39:29.361354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:29.362949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:29.439814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2b64b-3684-464b-8361-6a7789d61624 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:29.445062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:29.445680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:29.445962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:29.446338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:29.456498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:29.459566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:29.468830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1148/2274] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:39:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:39:30.045967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915630-4a40-43b1-87f2-278039f46900 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:30.046288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915630-4a40-43b1-87f2-278039f46900 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:30.046595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75915630-4a40-43b1-87f2-278039f46900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:30.067138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-75915630-4a40-43b1-87f2-278039f46900 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:30.068383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-75915630-4a40-43b1-87f2-278039f46900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:30.075571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1127/2275] 10.222.0.91 () {58 vars in 967 bytes} [Wed Aug 24 03:39:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:30.090221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:30.102154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09554ac-bda6-4a83-b467-25c10d4693bc service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:30.107178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:30.107773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:30.108044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:30.108311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:30.116874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:30.118922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:30.127585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1149/2276] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:39:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:39:30.382647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16984f-0a85-48de-8c0e-cd697b537408 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:30.383168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16984f-0a85-48de-8c0e-cd697b537408 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:30.383168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b16984f-0a85-48de-8c0e-cd697b537408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:30.402080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4b16984f-0a85-48de-8c0e-cd697b537408 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:30.403373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4b16984f-0a85-48de-8c0e-cd697b537408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:30.411116 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1128/2277] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:39:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:30.423908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:30.434347 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df1e1a4-09ba-4a7a-9bb9-7452964562ce service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:30.439651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:30.440242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:30.440521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:30.440792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:30.448656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:30.450225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:30.458402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1150/2278] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:39:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:39:30.802372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:30.802661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:30.802937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:30.822771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:30.823984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:30.831869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1129/2279] 10.222.0.91 () {58 vars in 967 bytes} [Wed Aug 24 03:39:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:30.848415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:30.859303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe234b-1bed-470a-addf-b8e4ed025824 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:30.867081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823c306-c5b5-4a8c-b2d0-72a766e96b3a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:30.867977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c306-c5b5-4a8c-b2d0-72a766e96b3a service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:30.868375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c306-c5b5-4a8c-b2d0-72a766e96b3a service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:30.868805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823c306-c5b5-4a8c-b2d0-72a766e96b3a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:30.883925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823c306-c5b5-4a8c-b2d0-72a766e96b3a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:30.886336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f823c306-c5b5-4a8c-b2d0-72a766e96b3a service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:30.902676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1151/2280] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:39:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:39:34.121744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:34.123250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:34.190744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1faac832-aa66-4fcc-9677-b7f04e717af6 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:34.197067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:34.197696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:34.197977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:34.198276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] PATH_INFO: `/v3/users/ab3dbf21871d4ee284f3c155a1b70ce7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:34.199434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab3dbf21871d4ee284f3c155a1b70ce7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:34.201374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:34.250713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab3dbf21871d4ee284f3c155a1b70ce7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.251866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab3dbf21871d4ee284f3c155a1b70ce7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.252472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab3dbf21871d4ee284f3c155a1b70ce7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.259468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab3dbf21871d4ee284f3c155a1b70ce7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.268481 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab3dbf21871d4ee284f3c155a1b70ce7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.291254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1130/2281] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:39:34 2022] DELETE /identity/v3/users/ab3dbf21871d4ee284f3c155a1b70ce7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:34.301033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:34.301531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:34.301627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:34.322610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:34.323866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:34.363693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:34.391531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1152/2282] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:34.401395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:34.402887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:34.411052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:34.472330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d2304-154f-4cb4-8f5d-053961f72a80 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:34.476475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5d7656-b04c-479f-ad43-a8da317aa583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:34.477095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d7656-b04c-479f-ad43-a8da317aa583 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:34.477367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d7656-b04c-479f-ad43-a8da317aa583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:34.477634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5d7656-b04c-479f-ad43-a8da317aa583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:34.491242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d7656-b04c-479f-ad43-a8da317aa583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:34.493699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5d7656-b04c-479f-ad43-a8da317aa583 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:34.509533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1131/2283] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:39:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:39:34.762486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:34.764268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:34.777438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621a8de6-59b7-4cd8-80b5-57d86ba6092f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:34.782987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:34.783628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:34.783903 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:34.784193 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] PATH_INFO: `/v3/projects/76c2e8ca4e594275a764e2bfd287af9d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:34.785544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=76c2e8ca4e594275a764e2bfd287af9d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:34.787498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:34.846323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76c2e8ca4e594275a764e2bfd287af9d', 'request_id': 'req-2654f7c6-8e26-4e37-b988-a921b89bdf1a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.852707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76c2e8ca4e594275a764e2bfd287af9d', 'request_id': 'req-2654f7c6-8e26-4e37-b988-a921b89bdf1a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.863021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76c2e8ca4e594275a764e2bfd287af9d', 'request_id': 'req-2654f7c6-8e26-4e37-b988-a921b89bdf1a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.884636 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] The token cache is being invalidate because project 76c2e8ca4e594275a764e2bfd287af9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:39:34.885015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.885788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2654f7c6-8e26-4e37-b988-a921b89bdf1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:39:34.888452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1153/2284] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:39:34 2022] DELETE /identity/v3/projects/76c2e8ca4e594275a764e2bfd287af9d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:34.897224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:34.897710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:34.898286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:34.923285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:34.925685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:35.006744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1132/2285] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:39:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:39:35.014876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.015157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.015452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.036182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:35.037411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:35.046944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1154/2286] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:35.051415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.061286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dbda13-c4ee-40a3-a8eb-864310a85045 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.067281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.067893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.068209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.068463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.069115 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:39:35.069598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.071262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.085071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1133/2287] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:39:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:39:35.089986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.099746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfce957-d851-45e1-b7a8-1d4c1c8ecea5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.105020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.105638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.105935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.106218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.106848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.108757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.138065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1155/2288] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:39:35 2022] POST /identity/v3/projects => generated 379 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:39:35.142978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.153808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bee220-adfe-4052-b948-482e42275b22 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.216181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.216795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.217104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.217398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.218038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.220566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.245081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1134/2289] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:39:35 2022] POST /identity/v3/users => generated 290 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:39:35.249355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.260037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc377ee-1dc8-4e7d-a47c-77ba27ca34f5 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.265180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.265880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.266165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.267025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.267817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.269296 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.281755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1156/2290] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:39:35.286545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.296882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f852f19-0d30-4e98-831d-f51d7fdd8947 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.301950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abcfa78-14cd-4643-8ed1-1733b12b5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.302591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcfa78-14cd-4643-8ed1-1733b12b5276 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.302877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcfa78-14cd-4643-8ed1-1733b12b5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.303152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abcfa78-14cd-4643-8ed1-1733b12b5276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.303800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcfa78-14cd-4643-8ed1-1733b12b5276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.305441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abcfa78-14cd-4643-8ed1-1733b12b5276 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.316220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1135/2291] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:39:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:39:35.320506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.332132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da45a916-1919-4064-9b9b-9581148d48c3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.336998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.337660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.337942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.338302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] PATH_INFO: `/v3/projects/0fe6a4ca8c984b2aa4df4c47b4c1d03b/users/b9f929b986e645109db65651c8ff1502/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.355096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fe6a4ca8c984b2aa4df4c47b4c1d03b, user_id=b9f929b986e645109db65651c8ff1502, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.358629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.373026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1157/2292] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:39:35 2022] PUT /identity/v3/projects/0fe6a4ca8c984b2aa4df4c47b4c1d03b/users/b9f929b986e645109db65651c8ff1502/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:35.378477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc831db-75f6-4159-b6a4-4b336102ff1d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.378841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc831db-75f6-4159-b6a4-4b336102ff1d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.379188 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc831db-75f6-4159-b6a4-4b336102ff1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.400499 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2cc831db-75f6-4159-b6a4-4b336102ff1d None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:35.401715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2cc831db-75f6-4159-b6a4-4b336102ff1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:35.419426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:35.485277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1136/2293] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:39:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:39:35.494713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:35.496431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:35.559896 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ffe16-c705-47c3-923a-e9ac0bec2304 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:35.564165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:35.564783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:35.565066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:35.565339 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:35.574653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:35.576114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:35.586084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1158/2294] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:39:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:39:42.264868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:42.265526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:42.265526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:42.314773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] MFA Rules not processed for user `b9f929b986e645109db65651c8ff1502`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:42.317107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:42.467269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1137/2295] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:39:42 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:42.467561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:42.475644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:42.483192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f116946-c382-441f-8621-e171ad81833d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:42.490286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:42.491262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:42.491843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:42.492420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:42.493333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:42.495315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:42.532302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1159/2296] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:39:42 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:39:42.536709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:42.549664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbfb4a3-1be6-4c56-b2ed-1283ef77f959 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:42.634029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:42.634778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:42.635099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:42.635379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:42.636036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:42.638174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:42.671495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1138/2297] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:39:42 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:39:42.676345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:42.689384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad2a49-1009-4941-acd2-e9f994928f45 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:42.695652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:42.696539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:42.697049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:42.697491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:42.698427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:42.700443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:42.719045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:42.720510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1160/2298] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:39:42.732588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c4e5c-7770-4070-b385-4e159213c7f1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:42.738810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:42.739605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:42.739990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:42.740521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] PATH_INFO: `/v3/projects/42ba9acaf363498ea12cedff000a6005/users/2a30c7b692d242869e99c96e9a04a567/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:42.768002 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ba9acaf363498ea12cedff000a6005, user_id=2a30c7b692d242869e99c96e9a04a567, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:42.772808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:42.825522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1139/2299] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:39:42 2022] PUT /identity/v3/projects/42ba9acaf363498ea12cedff000a6005/users/2a30c7b692d242869e99c96e9a04a567/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:42.833815 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:42.846307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090cf50c-479e-4329-9a02-5a5b41f9de7c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:42.930047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0518e8-7bb3-43fc-833f-2243f1c5105d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:42.931161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0518e8-7bb3-43fc-833f-2243f1c5105d admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:42.931662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0518e8-7bb3-43fc-833f-2243f1c5105d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:42.932151 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0518e8-7bb3-43fc-833f-2243f1c5105d admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:42.933045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0518e8-7bb3-43fc-833f-2243f1c5105d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:42.934934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0518e8-7bb3-43fc-833f-2243f1c5105d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:42.964882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1161/2300] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:39:42 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:39:42.971022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:42.991150 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdebc04-1e65-4c31-b072-78df87188039 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:43.005523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:43.006732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:43.007075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:43.007418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] PATH_INFO: `/v3/projects/42ba9acaf363498ea12cedff000a6005/users/2a30c7b692d242869e99c96e9a04a567/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:43.012221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ba9acaf363498ea12cedff000a6005, user_id=2a30c7b692d242869e99c96e9a04a567, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:43.015176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:43.022505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:43.055132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1140/2301] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:39:42 2022] PUT /identity/v3/projects/42ba9acaf363498ea12cedff000a6005/users/2a30c7b692d242869e99c96e9a04a567/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:39:49.957424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718c22-7a6e-4f21-b8c5-19469f220f63 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:49.958292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718c22-7a6e-4f21-b8c5-19469f220f63 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:49.958292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f718c22-7a6e-4f21-b8c5-19469f220f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:49.994702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3f718c22-7a6e-4f21-b8c5-19469f220f63 None None] MFA Rules not processed for user `2a30c7b692d242869e99c96e9a04a567`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:49.996016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3f718c22-7a6e-4f21-b8c5-19469f220f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:50.127199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1162/2302] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:39:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:50.142516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:50.144931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:50.240221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febdf23-3083-40a7-a763-dab434f11864 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:50.244306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:50.244795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:50.245094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:50.245401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:50.323121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:50.323121 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:50.342348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1141/2303] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:39:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:39:50.510964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01bafd-1acf-411f-8dfb-6d15a3d4d216 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:50.511270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01bafd-1acf-411f-8dfb-6d15a3d4d216 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:50.511551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce01bafd-1acf-411f-8dfb-6d15a3d4d216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:50.537042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ce01bafd-1acf-411f-8dfb-6d15a3d4d216 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:39:50.538563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ce01bafd-1acf-411f-8dfb-6d15a3d4d216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:50.603024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:39:50.616110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1163/2304] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:39:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:39:50.629662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:39:50.631258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:39:50.723906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776e7c8-a1a4-48d6-bc61-5df80c49a729 service glance] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:39:50.729650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ede757-04b6-4cd3-8c15-daf495df8c4f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:39:50.730306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ede757-04b6-4cd3-8c15-daf495df8c4f service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:39:50.730657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ede757-04b6-4cd3-8c15-daf495df8c4f service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:39:50.730963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ede757-04b6-4cd3-8c15-daf495df8c4f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:39:50.741011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ede757-04b6-4cd3-8c15-daf495df8c4f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:39:50.742713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ede757-04b6-4cd3-8c15-daf495df8c4f service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:39:50.752322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1142/2305] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:39:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:40:00.387227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:00.387875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:00.388018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:00.408409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] MFA Rules not processed for user `b9f929b986e645109db65651c8ff1502`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:00.409641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:00.417797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1164/2306] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:00.748578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=82 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:01.397142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:01.397716 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:01.398203 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:01.454644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:01.456078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:01.466402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1143/2307] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:40:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:01.480090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:01.494214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ffc43-400c-4552-9781-d6cd8d519bf0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:01.500535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6835b745-584f-457d-92b8-c3ca65609103 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:01.501858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835b745-584f-457d-92b8-c3ca65609103 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:01.502448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835b745-584f-457d-92b8-c3ca65609103 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:01.502709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6835b745-584f-457d-92b8-c3ca65609103 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:01.515054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835b745-584f-457d-92b8-c3ca65609103 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:01.517892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6835b745-584f-457d-92b8-c3ca65609103 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:01.531390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1165/2308] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:40:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:40:02.719118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:02.722029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:02.839929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52585fa4-0c83-414b-8ad7-bf4b484bb84b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:02.845581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:02.847361 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:02.847799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:02.848181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] PATH_INFO: `/v3/users/8c99c4bb79d84331964edba016d2a0c4` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:02.850598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c99c4bb79d84331964edba016d2a0c4)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:02.852720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:02.943915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c99c4bb79d84331964edba016d2a0c4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:02.946447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c99c4bb79d84331964edba016d2a0c4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:02.947890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c99c4bb79d84331964edba016d2a0c4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:02.965433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c99c4bb79d84331964edba016d2a0c4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:02.987389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-758c4d8e-3829-4492-b32a-77f66411bfd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c99c4bb79d84331964edba016d2a0c4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:03.012586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1144/2309] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:02 2022] DELETE /identity/v3/users/8c99c4bb79d84331964edba016d2a0c4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:03.018011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:03.018547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:03.019330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:03.041093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:03.055780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:03.168154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:03.168855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:03.169341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:03.209473 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:03.210117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1166/2310] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:03.211714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:03.245999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:03.252620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:03.356644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1145/2311] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:40:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:03.367035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:03.368729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:03.419335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:03.419549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8468a-fd21-45dc-8506-d4fa148152ff None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:03.425480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:03.427755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:03.427930 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:03.428185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:03.442434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:03.444375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:03.460824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1167/2312] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:03.517509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cc0d96-4540-4886-9129-66125ca6700d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:03.539070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:03.539821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:03.540388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:03.540807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:03.563523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:03.568890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:03.591805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1146/2313] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:40:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:40:04.211162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:04.213493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:04.276591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a2595-e4da-4771-92e0-fee1d4e1b64a service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:04.321444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:04.322115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:04.322426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:04.322703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] PATH_INFO: `/v3/projects/5d72f8ab73384482a4a2216939836da2` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:04.324536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d72f8ab73384482a4a2216939836da2)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:04.326524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:04.430958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d72f8ab73384482a4a2216939836da2', 'request_id': 'req-c5064ce1-983c-4996-9a0c-b427e6bc2628'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.440574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d72f8ab73384482a4a2216939836da2', 'request_id': 'req-c5064ce1-983c-4996-9a0c-b427e6bc2628'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.449773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d72f8ab73384482a4a2216939836da2', 'request_id': 'req-c5064ce1-983c-4996-9a0c-b427e6bc2628'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.469202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] The token cache is being invalidate because project 5d72f8ab73384482a4a2216939836da2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:04.470025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.470942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.472831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1168/2314] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:04 2022] DELETE /identity/v3/projects/5d72f8ab73384482a4a2216939836da2 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:04.477035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:04.511770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ab720-2f54-4d1d-8a59-aec33b040a9d service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:04.596846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:04.597597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:04.597943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:04.598349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] PATH_INFO: `/v3/users/87374d6cb4da416bae5da67d0ae2edfa` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:04.600092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] RBAC: Authorizing `identity:delete_user(user_id=87374d6cb4da416bae5da67d0ae2edfa)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:04.602819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:04.717944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87374d6cb4da416bae5da67d0ae2edfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.718745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87374d6cb4da416bae5da67d0ae2edfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.731237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87374d6cb4da416bae5da67d0ae2edfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.742669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87374d6cb4da416bae5da67d0ae2edfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.755196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b2e67bfa-0271-4e6f-b7c2-7a1e0454138a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87374d6cb4da416bae5da67d0ae2edfa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:04.784574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1147/2315] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:04 2022] DELETE /identity/v3/users/87374d6cb4da416bae5da67d0ae2edfa => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:05.279547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:05.281047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:05.289839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:05.303951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5064ce1-983c-4996-9a0c-b427e6bc2628 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:05.400054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:05.400731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:05.401009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:05.401282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] PATH_INFO: `/v3/projects/94f17d978cee4805a776a222fe28f30d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:05.403492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94f17d978cee4805a776a222fe28f30d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:05.405593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:05.459683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f17d978cee4805a776a222fe28f30d', 'request_id': 'req-5e142913-97fe-46fe-85c5-9d5caeabb549'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:05.472163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f17d978cee4805a776a222fe28f30d', 'request_id': 'req-5e142913-97fe-46fe-85c5-9d5caeabb549'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:05.486671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f17d978cee4805a776a222fe28f30d', 'request_id': 'req-5e142913-97fe-46fe-85c5-9d5caeabb549'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:05.515953 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] The token cache is being invalidate because project 94f17d978cee4805a776a222fe28f30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:05.517520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:05.518681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e142913-97fe-46fe-85c5-9d5caeabb549 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:05.522146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1169/2316] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:05 2022] DELETE /identity/v3/projects/94f17d978cee4805a776a222fe28f30d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:05.528674 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:05.529705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:05.530148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:05.556893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:05.558236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:05.585024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:05.659611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1148/2317] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:05.667719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:05.668299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:05.668774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:05.708405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:05.711597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:05.721890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1170/2318] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:05.726270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:05.739620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e27c5a8-07ed-4bde-8655-277d07d92a40 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:05.748317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:05.749963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:05.751916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:05.754093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:05.756201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:40:05.757593 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:05.760387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:05.790153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1149/2319] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:40:05.794731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:05.807218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc3f772-10cf-4c37-9036-cb25c8c5758c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:05.816515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:05.817503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:05.817941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:05.818393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:05.819264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:05.821438 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:05.879885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1171/2320] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:05 2022] POST /identity/v3/projects => generated 409 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:05.884000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:05.903754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636dabba-fb95-41e2-8121-98b457fc6f6d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.020494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.021126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.021564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.021989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.022647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.024957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:06.074962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1150/2321] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:05 2022] POST /identity/v3/users => generated 305 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:06.080184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:06.121702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a7e312-a350-457a-9540-c28a8874115f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.129325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.130159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.130480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.130757 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.132553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.135047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:06.166050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1172/2322] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:06.167603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:06.179578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218d15c6-4ace-4e9f-9d24-ef94b197b0ec admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.184997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c7962e-958b-4db6-9885-1a331a6408e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.185991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7962e-958b-4db6-9885-1a331a6408e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.186431 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7962e-958b-4db6-9885-1a331a6408e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.186809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c7962e-958b-4db6-9885-1a331a6408e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.189192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7962e-958b-4db6-9885-1a331a6408e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.189680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c7962e-958b-4db6-9885-1a331a6408e7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:06.229825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1151/2323] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:06.231383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:06.247993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e62c31-adc6-4809-bdc0-7f8fdb376cf2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.256124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.257448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.257857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.258214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] PATH_INFO: `/v3/projects/06826a617feb447b8b5ad0f9f1ae648a/users/81c4e9a4610844b39520db2ce915347e/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.301487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06826a617feb447b8b5ad0f9f1ae648a, user_id=81c4e9a4610844b39520db2ce915347e, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.306674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:06.346921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1173/2324] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:06 2022] PUT /identity/v3/projects/06826a617feb447b8b5ad0f9f1ae648a/users/81c4e9a4610844b39520db2ce915347e/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:06.362871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.363467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.363946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.437420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] MFA Rules not processed for user `81c4e9a4610844b39520db2ce915347e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:06.438771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:06.559107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1152/2325] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:06.567661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:06.626487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:06.677855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d2f5-051c-466b-8230-5869e3b72e55 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.682750 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.683731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.683871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.684943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.684943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.686925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:06.749832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1174/2326] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:06 2022] POST /identity/v3/projects => generated 409 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:40:06.750080 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:06.767117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ba22-6eb2-49ba-b81c-a0ccfeae3739 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.801631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:06.883189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.883825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.884132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.884377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.885029 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.887191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:06.929142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:06.929638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1153/2327] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:06 2022] POST /identity/v3/users => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:06.944175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11abd19-92f9-4b65-9bc9-3d66134eabb2 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:06.957102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:06.957102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:06.957102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:06.957102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:06.978009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:06.986763 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:07.019287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1175/2328] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:40:07.023589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:07.043607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ef90f-5e58-49ba-99ca-279291f2af1f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:07.050365 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:07.051476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:07.051975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:07.052540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] PATH_INFO: `/v3/projects/f7c6f86e471044fa9427935b662ece1b/users/b26503c2bdbe4b6687df9b94adfc4ec4/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:07.102295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c6f86e471044fa9427935b662ece1b, user_id=b26503c2bdbe4b6687df9b94adfc4ec4, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:07.106210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:07.138994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1154/2329] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:07 2022] PUT /identity/v3/projects/f7c6f86e471044fa9427935b662ece1b/users/b26503c2bdbe4b6687df9b94adfc4ec4/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:07.140532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:07.200727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3a8661-5a53-46dd-833a-aba41d6ce8d4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:07.325472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee043e-e82f-4570-800f-aa59ef0fe8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:07.326315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee043e-e82f-4570-800f-aa59ef0fe8e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:07.326733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee043e-e82f-4570-800f-aa59ef0fe8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:07.327074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee043e-e82f-4570-800f-aa59ef0fe8e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:07.327947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee043e-e82f-4570-800f-aa59ef0fe8e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:07.330394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee043e-e82f-4570-800f-aa59ef0fe8e7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:07.353324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:07.355170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1176/2330] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:07 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:07.369348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ae53f-6a7e-4bf5-be69-4f3936ee6020 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:07.375967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:07.377054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:07.380760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:07.381822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] PATH_INFO: `/v3/projects/f7c6f86e471044fa9427935b662ece1b/users/b26503c2bdbe4b6687df9b94adfc4ec4/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:07.383911 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7c6f86e471044fa9427935b662ece1b, user_id=b26503c2bdbe4b6687df9b94adfc4ec4, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:07.387566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:07.415449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1155/2331] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:07 2022] PUT /identity/v3/projects/f7c6f86e471044fa9427935b662ece1b/users/b26503c2bdbe4b6687df9b94adfc4ec4/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:07.427805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:07.428110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:07.428405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:07.505443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] MFA Rules not processed for user `b26503c2bdbe4b6687df9b94adfc4ec4`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:07.506872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:07.631365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1177/2332] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:07 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:07.658263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:07.660176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:07.882961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7558b611-0f43-421e-b4d6-46ea9859eb95 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:07.909224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4602af-e0b2-483a-9be9-c400563bd25a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:07.909224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4602af-e0b2-483a-9be9-c400563bd25a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:07.909224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4602af-e0b2-483a-9be9-c400563bd25a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:07.909224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4602af-e0b2-483a-9be9-c400563bd25a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:07.932763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4602af-e0b2-483a-9be9-c400563bd25a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:07.934277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4602af-e0b2-483a-9be9-c400563bd25a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:07.978398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1156/2333] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:07 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:08.313751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:08.334128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e499177-1644-42d0-acbf-e8dc5bd81b74 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:08.342142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1f2912-ea7e-48d6-b0a9-a59252673a42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:08.342956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f2912-ea7e-48d6-b0a9-a59252673a42 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:08.343394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f2912-ea7e-48d6-b0a9-a59252673a42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:08.343676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1f2912-ea7e-48d6-b0a9-a59252673a42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:08.385273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:08.445756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f2912-ea7e-48d6-b0a9-a59252673a42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:08.447465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1f2912-ea7e-48d6-b0a9-a59252673a42 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:08.488200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1178/2334] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:08 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:14.999359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:15.000129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:15.000991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:15.057839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] MFA Rules not processed for user `76c0df49fa9342778a56eae098b2f2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:15.061800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:15.175373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1157/2335] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:18.484897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=17 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:20.653877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:20.654874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:20.655318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:20.677620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:20.678940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:20.769057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1179/2336] 10.222.0.91 () {58 vars in 967 bytes} [Wed Aug 24 03:40:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:20.783648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:20.788840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:20.835474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:20.926363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e80103c-2439-4b01-a0e4-afac6a2d6660 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:20.934409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:20.935306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:20.935744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:20.936176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:20.947552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:20.949529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:20.964793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1158/2337] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:40:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:40:23.198593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:23.200283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:23.308299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43078fcc-c726-45b3-ace4-4ebeff067f40 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:23.313095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:23.313824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.314107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.314547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] PATH_INFO: `/v3/users/5439318804a745d98ee897fb511ed9af` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.315782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5439318804a745d98ee897fb511ed9af)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:23.318350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:23.429027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5439318804a745d98ee897fb511ed9af'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.429887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5439318804a745d98ee897fb511ed9af'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.430797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5439318804a745d98ee897fb511ed9af'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.446309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5439318804a745d98ee897fb511ed9af'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.458281 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5439318804a745d98ee897fb511ed9af'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.485025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1180/2338] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:23 2022] DELETE /identity/v3/users/5439318804a745d98ee897fb511ed9af => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:23.534852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:23.545940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e1cff2-b486-404b-ba48-65e43708dc99 service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:23.617928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:23.618782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.619132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.619461 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] PATH_INFO: `/v3/users/81c4e9a4610844b39520db2ce915347e` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.620667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c4e9a4610844b39520db2ce915347e)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:23.622659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:23.680149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c4e9a4610844b39520db2ce915347e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.681518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c4e9a4610844b39520db2ce915347e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.682594 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c4e9a4610844b39520db2ce915347e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.690169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:23.692195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:23.698875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c4e9a4610844b39520db2ce915347e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.709831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-971e6b3b-f20a-47a8-ac19-bf2c0c3c1833 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c4e9a4610844b39520db2ce915347e'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.711913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b58693f-c0d8-4b39-8f9d-3c81b126626b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:23.718796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:23.719258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.719535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.719900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] PATH_INFO: `/v3/projects/97e2f9bd5a7b4fc8828847d518968bfd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.721195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e2f9bd5a7b4fc8828847d518968bfd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:23.723769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:23.736069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1159/2339] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:23 2022] DELETE /identity/v3/users/81c4e9a4610844b39520db2ce915347e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:23.740463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.740749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.741024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.748384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:23.763151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:23.764403 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:23.782577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e2f9bd5a7b4fc8828847d518968bfd', 'request_id': 'req-6ece99bf-f78e-4aac-b3d1-931944c36339'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.789211 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e2f9bd5a7b4fc8828847d518968bfd', 'request_id': 'req-6ece99bf-f78e-4aac-b3d1-931944c36339'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.794041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:23.796539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e2f9bd5a7b4fc8828847d518968bfd', 'request_id': 'req-6ece99bf-f78e-4aac-b3d1-931944c36339'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.810472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] The token cache is being invalidate because project 97e2f9bd5a7b4fc8828847d518968bfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:23.810822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.811872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6ece99bf-f78e-4aac-b3d1-931944c36339 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:23.813870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1181/2340] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:23 2022] DELETE /identity/v3/projects/97e2f9bd5a7b4fc8828847d518968bfd => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:23.819615 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208853b2-ac89-4417-9d85-35230abbb79a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.819900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208853b2-ac89-4417-9d85-35230abbb79a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.820179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208853b2-ac89-4417-9d85-35230abbb79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.832418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1160/2341] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:23.840380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-208853b2-ac89-4417-9d85-35230abbb79a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:23.841610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-208853b2-ac89-4417-9d85-35230abbb79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:23.843146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:23.845248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:23.850603 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1182/2342] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:23.858072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.858391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.858665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.883304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:23.884542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:23.893256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1183/2343] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:23.897489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:23.907525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac51271-77a9-4eb5-a39c-ee640ba31015 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:23.911581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d80edd7-04b9-499e-bd0d-1943653f0543 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:23.912344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:23.912956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.913271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.913538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.914210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:40:23.914596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:23.915434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:23.916082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.916187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:23.916395 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.916654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.927240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:23.928739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:23.932756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1184/2344] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:40:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:40:23.933996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:23.939649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1161/2345] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:40:23.944736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d16cc4-0815-4fbf-bcd2-3090faf09e4b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:23.950079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:23.950722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:23.951001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:23.951272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:23.951898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:23.953841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:23.989384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1185/2346] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:23 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:40:23.993667 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.003438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1726879-1219-488d-a8c2-ba880458c227 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.065604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.066240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.066524 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.066851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.067503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.069452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.094923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1162/2347] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:23 2022] POST /identity/v3/users => generated 282 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:24.103658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.115487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9289dddb-a328-4330-b2e9-541f72cc1853 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.120370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.120975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.121251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.121590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.122261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.123872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.135203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1186/2348] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:24.139133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.150913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18de751-8c33-4371-b3d6-3dfd4f6ea847 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.158206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48741388-eebb-4ed0-87a3-723732082681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.158966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48741388-eebb-4ed0-87a3-723732082681 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.159360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48741388-eebb-4ed0-87a3-723732082681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.159739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48741388-eebb-4ed0-87a3-723732082681 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.160526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48741388-eebb-4ed0-87a3-723732082681 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.162444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48741388-eebb-4ed0-87a3-723732082681 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.173736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1163/2349] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:24.178286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.191089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79e93af-e0d5-4a16-96cc-2aec39befcbf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.195954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.196703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.196986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.197260 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] PATH_INFO: `/v3/projects/8c0b7bcc238c4130a25594dc1becca2d/users/ba24c2a2c8a849c0a3c8e19964017887/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.213092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c0b7bcc238c4130a25594dc1becca2d, user_id=ba24c2a2c8a849c0a3c8e19964017887, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.215814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.233118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1187/2350] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:24 2022] PUT /identity/v3/projects/8c0b7bcc238c4130a25594dc1becca2d/users/ba24c2a2c8a849c0a3c8e19964017887/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:24.243993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.244680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ac718-b707-4255-9606-78660d20d5dd None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.244940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ac718-b707-4255-9606-78660d20d5dd None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.245213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014ac718-b707-4255-9606-78660d20d5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.247085 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:24.280959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-014ac718-b707-4255-9606-78660d20d5dd None None] MFA Rules not processed for user `ba24c2a2c8a849c0a3c8e19964017887`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:24.283067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-014ac718-b707-4255-9606-78660d20d5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:24.333694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:24.346733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abfb514-eafa-4b46-a80b-5b346beb3bae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.351476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.352105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.352384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.352654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] PATH_INFO: `/v3/projects/06826a617feb447b8b5ad0f9f1ae648a` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.353914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06826a617feb447b8b5ad0f9f1ae648a)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.355867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.359725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1164/2351] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:24 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:24.371069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ac718-b707-4255-9606-78660d20d5dd None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.372437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-014ac718-b707-4255-9606-78660d20d5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:24.387821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=8,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:24.405380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06826a617feb447b8b5ad0f9f1ae648a', 'request_id': 'req-c6b9f141-a219-4501-a1bc-7c38647c5615'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.410928 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06826a617feb447b8b5ad0f9f1ae648a', 'request_id': 'req-c6b9f141-a219-4501-a1bc-7c38647c5615'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.417270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06826a617feb447b8b5ad0f9f1ae648a', 'request_id': 'req-c6b9f141-a219-4501-a1bc-7c38647c5615'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.432084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] The token cache is being invalidate because project 06826a617feb447b8b5ad0f9f1ae648a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:24.432493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.433138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.434751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1188/2352] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:24 2022] DELETE /identity/v3/projects/06826a617feb447b8b5ad0f9f1ae648a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:24.434904 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014ac718-b707-4255-9606-78660d20d5dd None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.439029 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:24.440358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:24.489350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.489956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.490274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.490529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.492397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:24.515054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b9f141-a219-4501-a1bc-7c38647c5615 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:24.521046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:24.521796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:24.522353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:24.523241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] PATH_INFO: `/v3/users/b26503c2bdbe4b6687df9b94adfc4ec4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:24.524521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b26503c2bdbe4b6687df9b94adfc4ec4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.526638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.564582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:24.566384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:24.579820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1165/2353] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:24 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:40:24.616097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b26503c2bdbe4b6687df9b94adfc4ec4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.617040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b26503c2bdbe4b6687df9b94adfc4ec4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.617821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b26503c2bdbe4b6687df9b94adfc4ec4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.634814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b26503c2bdbe4b6687df9b94adfc4ec4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.654794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1cb5da74-7ca2-4482-9c74-c51b1cc13c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b26503c2bdbe4b6687df9b94adfc4ec4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:24.671297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1189/2354] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:24 2022] DELETE /identity/v3/users/b26503c2bdbe4b6687df9b94adfc4ec4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:25.003243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.017405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb084e70-9d5f-4199-abc4-b50b3ba99a2f service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.071018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:25.081892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.082657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.082882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.083187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] PATH_INFO: `/v3/projects/f7c6f86e471044fa9427935b662ece1b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.084375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7c6f86e471044fa9427935b662ece1b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.086387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.138464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7c6f86e471044fa9427935b662ece1b', 'request_id': 'req-5ea74b50-7d70-47c4-93a2-46d0597a0131'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:25.144915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7c6f86e471044fa9427935b662ece1b', 'request_id': 'req-5ea74b50-7d70-47c4-93a2-46d0597a0131'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:25.152508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7c6f86e471044fa9427935b662ece1b', 'request_id': 'req-5ea74b50-7d70-47c4-93a2-46d0597a0131'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:25.165422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] The token cache is being invalidate because project f7c6f86e471044fa9427935b662ece1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:25.165712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:25.166389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-5ea74b50-7d70-47c4-93a2-46d0597a0131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:25.167888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1166/2355] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:25 2022] DELETE /identity/v3/projects/f7c6f86e471044fa9427935b662ece1b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:25.172819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.173156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.173412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.194516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:25.195738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:25.225231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:25.258448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1190/2356] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:25.266871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.267252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.267427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.286918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:25.288066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:25.295854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1167/2357] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:25.300425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.310328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af35fbf7-5a59-4294-89bd-c6667448065b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.315435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.316028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.316305 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.316573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.317217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:40:25.317520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.319123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.331157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1191/2358] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:40:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:40:25.335508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.345406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db779d09-36c1-47b7-aaef-b94b6a53b494 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.350334 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.350942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.351222 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.351489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.352130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.354100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.387474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1168/2359] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:25.388972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.400182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc6e140-0df7-4ad1-8423-c834d1191404 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.461834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.462486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.462846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.463045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.463683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.465660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.489311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1192/2360] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:25.493686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.505823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7fa161-436f-4814-80c0-ac9b6129317b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.510845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.511478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.511759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.512052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.512750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.514422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.524561 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1169/2361] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:25.528883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.539321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d899d2-7eca-4816-978f-1b2aa6d697a4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.544270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbe7d0b-a38d-48c6-bd13-5e3d22e771ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.544996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe7d0b-a38d-48c6-bd13-5e3d22e771ab admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.545307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe7d0b-a38d-48c6-bd13-5e3d22e771ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.545659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbe7d0b-a38d-48c6-bd13-5e3d22e771ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.546389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe7d0b-a38d-48c6-bd13-5e3d22e771ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.548580 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbe7d0b-a38d-48c6-bd13-5e3d22e771ab admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.560002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1193/2362] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:25.565102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.575089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916435fd-0214-4c09-bc70-4adccb1f963a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.580682 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.581372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.581706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.582315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] PATH_INFO: `/v3/projects/026bbdcb9b484fa7b1475ba8c984f1f5/users/b8cbc04f40df494c9f789ff40da076a0/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.597971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] RBAC: Authorizing `identity:create_grant(project_id=026bbdcb9b484fa7b1475ba8c984f1f5, user_id=b8cbc04f40df494c9f789ff40da076a0, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.600719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.615374 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1170/2363] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:25 2022] PUT /identity/v3/projects/026bbdcb9b484fa7b1475ba8c984f1f5/users/b8cbc04f40df494c9f789ff40da076a0/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:40:25.627460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.627744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.628015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.657728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] MFA Rules not processed for user `b8cbc04f40df494c9f789ff40da076a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:25.659013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:25.731565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1194/2364] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:25.736012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.801074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b07999b-4289-4503-b804-b846a414e808 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.805778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.806417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.806695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.806961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.807710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.809521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.836495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1171/2365] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:25.840586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.852170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1248d364-284d-4d18-b9ab-c3b526ed30a8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.879203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:25.911643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.912263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.912543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.912816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.913452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.915427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.939025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1195/2366] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:25 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:25.943141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:25.951170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:25.962623 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e88a69-ebb6-4fd7-a032-7fbe34bb3735 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:25.969675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:25.970576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:25.970867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:25.971422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:25.972406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:25.975309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:25.988199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1172/2367] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:25 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:25.992490 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:26.002794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff58b0-bd66-4a25-b01b-8550f7a53813 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:26.007675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:26.008431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:26.008792 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:26.009118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] PATH_INFO: `/v3/projects/4557252be3e04cb29906a63e169404d3/users/f7f08cb29abe47ea982ce0667c534a0b/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:26.026021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4557252be3e04cb29906a63e169404d3, user_id=f7f08cb29abe47ea982ce0667c534a0b, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:26.028828 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:26.044403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1196/2368] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:25 2022] PUT /identity/v3/projects/4557252be3e04cb29906a63e169404d3/users/f7f08cb29abe47ea982ce0667c534a0b/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:26.049836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:26.065143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d720afd-b90d-4a12-975d-6e77eb9c0126 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:26.137318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577328e3-59f1-4bec-b8b8-774110becce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:26.137929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577328e3-59f1-4bec-b8b8-774110becce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:26.138236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577328e3-59f1-4bec-b8b8-774110becce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:26.138516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577328e3-59f1-4bec-b8b8-774110becce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:26.139165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577328e3-59f1-4bec-b8b8-774110becce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:26.140773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577328e3-59f1-4bec-b8b8-774110becce9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:26.150680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1173/2369] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:26 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:26.154874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:26.165711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee25023e-cf40-4fc5-bd86-2389dbe85afd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:26.170456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:26.171110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:26.171392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:26.171672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] PATH_INFO: `/v3/projects/4557252be3e04cb29906a63e169404d3/users/f7f08cb29abe47ea982ce0667c534a0b/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:26.174173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4557252be3e04cb29906a63e169404d3, user_id=f7f08cb29abe47ea982ce0667c534a0b, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:26.177078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:26.188346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1197/2370] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:26 2022] PUT /identity/v3/projects/4557252be3e04cb29906a63e169404d3/users/f7f08cb29abe47ea982ce0667c534a0b/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:40:26.198487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:26.198773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:26.199049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:26.226902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] MFA Rules not processed for user `f7f08cb29abe47ea982ce0667c534a0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:26.228142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:26.297725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1174/2371] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:26 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:26.308731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:26.310461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:26.368567 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1424f10-2130-4efe-bec0-d3ca066d6ad7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:26.372272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c023594-877e-4503-9acc-bbaf07e9345e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:26.372895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c023594-877e-4503-9acc-bbaf07e9345e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:26.373168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c023594-877e-4503-9acc-bbaf07e9345e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:26.373436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c023594-877e-4503-9acc-bbaf07e9345e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:26.382905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c023594-877e-4503-9acc-bbaf07e9345e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:26.384315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c023594-877e-4503-9acc-bbaf07e9345e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:26.393627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1198/2372] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:26 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:27.008346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:27.136642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ef5cc5-7556-440e-9a22-e56b2181a1ee None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:27.143695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:27.144487 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:27.144917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:27.145261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] PATH_INFO: `/v3/users/b8cbc04f40df494c9f789ff40da076a0` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:27.158422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8cbc04f40df494c9f789ff40da076a0)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:27.158422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:27.185875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:27.241416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8cbc04f40df494c9f789ff40da076a0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:27.242368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8cbc04f40df494c9f789ff40da076a0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:27.243570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8cbc04f40df494c9f789ff40da076a0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:27.266665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8cbc04f40df494c9f789ff40da076a0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:27.279467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8cbc04f40df494c9f789ff40da076a0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:27.321685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1175/2373] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:27 2022] DELETE /identity/v3/users/b8cbc04f40df494c9f789ff40da076a0 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:27.327185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:27.328105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:27.328581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:27.356632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:27.358669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:27.454226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1199/2374] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:27.461576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:27.463108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:27.525262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81358340-ba66-4531-9507-6e7f31d55004 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:27.529485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:27.530508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:27.530508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:27.530737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:27.543643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:27.545130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:27.556530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1176/2375] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:28.379852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.381811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:28.395728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb80010c-b288-4908-9dfd-9924ffa6a936 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.401139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.401943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.402411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.402844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] PATH_INFO: `/v3/projects/026bbdcb9b484fa7b1475ba8c984f1f5` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.404180 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=026bbdcb9b484fa7b1475ba8c984f1f5)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.406486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.432534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:28.475782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '026bbdcb9b484fa7b1475ba8c984f1f5', 'request_id': 'req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.483276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '026bbdcb9b484fa7b1475ba8c984f1f5', 'request_id': 'req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.492666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '026bbdcb9b484fa7b1475ba8c984f1f5', 'request_id': 'req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.511401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] The token cache is being invalidate because project 026bbdcb9b484fa7b1475ba8c984f1f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:28.511808 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.512539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.514099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1200/2376] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:28 2022] DELETE /identity/v3/projects/026bbdcb9b484fa7b1475ba8c984f1f5 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:28.518779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:28.531142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d005f1-5e77-4b69-a88f-95bfbded86b3 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:28.595035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:28.596314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:28.596314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:28.596314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] PATH_INFO: `/v3/users/f7f08cb29abe47ea982ce0667c534a0b` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:28.597595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f08cb29abe47ea982ce0667c534a0b)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:28.599650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:28.655526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:28.665187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f08cb29abe47ea982ce0667c534a0b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.666219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f08cb29abe47ea982ce0667c534a0b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.666829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f08cb29abe47ea982ce0667c534a0b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.680336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f08cb29abe47ea982ce0667c534a0b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.696769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1187d17f-e059-4a63-9c05-946cf9def481 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f08cb29abe47ea982ce0667c534a0b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:28.730588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1177/2377] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:28 2022] DELETE /identity/v3/users/f7f08cb29abe47ea982ce0667c534a0b => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:29.109278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.111517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:29.139390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5f217b-5a88-48f3-9dd8-cceeef5c2a53 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.228208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.228877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.229167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.229455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] PATH_INFO: `/v3/projects/4557252be3e04cb29906a63e169404d3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.230725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4557252be3e04cb29906a63e169404d3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.232954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.291321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4557252be3e04cb29906a63e169404d3', 'request_id': 'req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.297487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4557252be3e04cb29906a63e169404d3', 'request_id': 'req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.304266 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4557252be3e04cb29906a63e169404d3', 'request_id': 'req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.317525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] The token cache is being invalidate because project 4557252be3e04cb29906a63e169404d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:29.317812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.318679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ee30b3b-81eb-42f0-875e-9cee43c8aaa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:29.320519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1201/2378] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:29 2022] DELETE /identity/v3/projects/4557252be3e04cb29906a63e169404d3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:29.326313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.326650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.326924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.351310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:29.352538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:29.424583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1178/2379] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:29.433996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.434552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.434704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.456290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:29.457576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:29.465880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1202/2380] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:29.470108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.479916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40f2b81-5f7f-4a79-95e6-76982dc61951 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.485784 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.486424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.486702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.486975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.487630 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:40:29.487930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.489880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.503213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1179/2381] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:40:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:40:29.507909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.518213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e82b3b-1c7f-4006-b867-c8dfca36e2b3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.523171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.523776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.524083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.524355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.524987 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.526935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.555939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1203/2382] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:29 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:40:29.559921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.570230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d45b19-d5d0-4115-ad80-f3bed1d287e5 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.627858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.628466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.628746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.629018 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.629664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.631611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.654948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1180/2383] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:29 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:29.659113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.669876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc0092-e6c9-44a0-a20f-af502a50089d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.675584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.676192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.676606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.676883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.677524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.679146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.681396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:29.691685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1204/2384] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:29.695471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.706655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50b020f-7991-45fb-9bd3-f9ab86422878 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.711647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb907f17-edca-4739-9a41-5178f4172084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.712409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907f17-edca-4739-9a41-5178f4172084 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.712817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907f17-edca-4739-9a41-5178f4172084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.713181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb907f17-edca-4739-9a41-5178f4172084 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.713925 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb907f17-edca-4739-9a41-5178f4172084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.715764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb907f17-edca-4739-9a41-5178f4172084 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.728093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1181/2385] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:29.732270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.742389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c88a6-f586-45b1-9a53-3e83743c8fd3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.747218 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.747874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.748158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.748437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] PATH_INFO: `/v3/projects/95fff4410db54bd980bb2064d0fd2645/users/461a94fa38ee4554b8c06ced6ae2940c/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.765583 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95fff4410db54bd980bb2064d0fd2645, user_id=461a94fa38ee4554b8c06ced6ae2940c, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.770020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:29.783363 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1205/2386] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:29 2022] PUT /identity/v3/projects/95fff4410db54bd980bb2064d0fd2645/users/461a94fa38ee4554b8c06ced6ae2940c/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:29.793847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.794135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.794447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.825411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] MFA Rules not processed for user `461a94fa38ee4554b8c06ced6ae2940c`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:29.826671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:29.837089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:29.896751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1182/2387] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:29.908306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:29.910313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:29.978468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9356ecd-a79b-4d06-a69c-f5860f58ed72 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:29.982137 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e18389-057d-4cd9-a788-81c57075b9a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:29.982772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e18389-057d-4cd9-a788-81c57075b9a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:29.983053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e18389-057d-4cd9-a788-81c57075b9a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:29.983324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e18389-057d-4cd9-a788-81c57075b9a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:29.992598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e18389-057d-4cd9-a788-81c57075b9a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:29.994348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e18389-057d-4cd9-a788-81c57075b9a5 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:30.003917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1206/2388] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:39.893764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=22 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:39.999303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=37,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:45.054042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:45.123558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0523799-53f2-41a6-bdce-644014df67d1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:45.128964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:45.129742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:45.130022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:45.130398 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] PATH_INFO: `/v3/users/461a94fa38ee4554b8c06ced6ae2940c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:45.131622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=461a94fa38ee4554b8c06ced6ae2940c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:45.133638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:45.198837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '461a94fa38ee4554b8c06ced6ae2940c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:45.200102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '461a94fa38ee4554b8c06ced6ae2940c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:45.200356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '461a94fa38ee4554b8c06ced6ae2940c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:45.209548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '461a94fa38ee4554b8c06ced6ae2940c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:45.218110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '461a94fa38ee4554b8c06ced6ae2940c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:45.241214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1183/2389] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:45 2022] DELETE /identity/v3/users/461a94fa38ee4554b8c06ced6ae2940c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:45.245765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:45.246550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:45.246876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:45.270242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:45.271579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:45.345958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1207/2390] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:45.358825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:45.360458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:45.435840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916adce0-4773-47e4-b036-537e4bafed9e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:45.440720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:45.441507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:45.442383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:45.442383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:45.453059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:45.455254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:45.465533 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1184/2391] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:40:45.763019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:45.776751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479b0e80-1866-4155-9e81-487d965b6547 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:45.780881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:45.781497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:45.781772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:45.782041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:45.784605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:45.837968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:45.839479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:45.856106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54390fd7-612c-4e73-97fa-96894421b7b1 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:45.881874 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:45.884331 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:45.885317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:45.885705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] PATH_INFO: `/v3/projects/95fff4410db54bd980bb2064d0fd2645` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:45.888184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:45.889728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d840a3ea-0fe7-46eb-ade1-cb353cedfbed service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:45.896152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95fff4410db54bd980bb2064d0fd2645)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:45.904783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:45.912098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1208/2392] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:40:45.941434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:45.990713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95fff4410db54bd980bb2064d0fd2645', 'request_id': 'req-f23136a8-7ee2-4970-9fd7-f6ea753ab066'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:45.996827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95fff4410db54bd980bb2064d0fd2645', 'request_id': 'req-f23136a8-7ee2-4970-9fd7-f6ea753ab066'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:46.007279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95fff4410db54bd980bb2064d0fd2645', 'request_id': 'req-f23136a8-7ee2-4970-9fd7-f6ea753ab066'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:46.031724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] The token cache is being invalidate because project 95fff4410db54bd980bb2064d0fd2645 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:40:46.032181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:46.033021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f23136a8-7ee2-4970-9fd7-f6ea753ab066 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:46.035024 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1185/2393] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:40:45 2022] DELETE /identity/v3/projects/95fff4410db54bd980bb2064d0fd2645 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:40:46.040910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.041387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.041847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.078448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:46.080626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:46.156195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1209/2394] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:46.165519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.166057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.166057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.212523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:46.215638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:46.227925 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1186/2395] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:46.232879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.273215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3baf8a-280a-4686-a566-7384efc0efbb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.285069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.288003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.288534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.288811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.289485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:40:46.289793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.291745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.305148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1210/2396] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:40:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:40:46.309313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.322394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab98da43-342f-4cab-a520-84e3b2823de2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.335006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.335529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.335790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.336072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.337345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.338789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.384764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1187/2397] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:40:46 2022] POST /identity/v3/projects => generated 375 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:46.399492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.411182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908ca15e-fec7-44fa-adca-1ae5564f28c1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.436353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:46.480095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.480711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.480993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.481311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.481957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.484096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.509652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1211/2398] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:40:46 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:40:46.511270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.521850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5e0915-0ef4-4b5a-b420-00934a40d225 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.527149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.527837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.528191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.528474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.529124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.530787 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.542172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1188/2399] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:46.546293 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.558103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc9ce5-8ab3-44c4-96e7-c866effd1cc9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.563217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c20e0-c496-4be4-a7d1-ea85bd68c989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.563795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c20e0-c496-4be4-a7d1-ea85bd68c989 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.564056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c20e0-c496-4be4-a7d1-ea85bd68c989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.564330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c20e0-c496-4be4-a7d1-ea85bd68c989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.564998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c20e0-c496-4be4-a7d1-ea85bd68c989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.566652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230c20e0-c496-4be4-a7d1-ea85bd68c989 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.578023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1212/2400] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:40:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:40:46.583041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.593392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe7bc0c-d76c-47ef-a886-c459d77c27c2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.598762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.599426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.599699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.600028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] PATH_INFO: `/v3/projects/6e86de3db260403d88f0e8924463627c/users/253375cbf3d041a5a8e71bf2163ee4b7/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.617597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e86de3db260403d88f0e8924463627c, user_id=253375cbf3d041a5a8e71bf2163ee4b7, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.620472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.635795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1189/2401] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:40:46 2022] PUT /identity/v3/projects/6e86de3db260403d88f0e8924463627c/users/253375cbf3d041a5a8e71bf2163ee4b7/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:40:46.646642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.647229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.647548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.687219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] MFA Rules not processed for user `253375cbf3d041a5a8e71bf2163ee4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:46.692415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:46.762086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1213/2402] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:40:46.775158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:46.777271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:46.842804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a21da6-2fb4-4c66-80f3-717bcea52ecf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:46.847373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01adcbc-51e7-4792-8c63-241979c265c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:46.847993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01adcbc-51e7-4792-8c63-241979c265c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:46.848273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01adcbc-51e7-4792-8c63-241979c265c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:46.848542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01adcbc-51e7-4792-8c63-241979c265c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:46.857730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01adcbc-51e7-4792-8c63-241979c265c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:46.859231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01adcbc-51e7-4792-8c63-241979c265c0 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:46.869162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1190/2403] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:40:56.759609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:56.865255 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=70,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:40:59.583862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:59.674527 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f027fac-82c5-4127-9f3a-71c22003dfa4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:40:59.683232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:40:59.684326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:59.684696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:59.685076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] PATH_INFO: `/v3/users/253375cbf3d041a5a8e71bf2163ee4b7` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:59.686488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=253375cbf3d041a5a8e71bf2163ee4b7)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:40:59.688513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:40:59.764543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253375cbf3d041a5a8e71bf2163ee4b7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:59.767247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253375cbf3d041a5a8e71bf2163ee4b7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:59.768761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253375cbf3d041a5a8e71bf2163ee4b7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:59.779710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253375cbf3d041a5a8e71bf2163ee4b7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:59.796743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253375cbf3d041a5a8e71bf2163ee4b7'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:40:59.825509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1214/2404] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:40:59 2022] DELETE /identity/v3/users/253375cbf3d041a5a8e71bf2163ee4b7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:40:59.827337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:40:59.827958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:40:59.827958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:40:59.857026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:40:59.858578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:40:59.982517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1191/2405] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:40:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:40:59.987320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:40:59.990037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:00.068157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a4575f-26f9-4e19-9662-881db013bb1b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:00.079972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbe70f-5715-4550-960d-c6cd38f4b449 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:00.081308 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe70f-5715-4550-960d-c6cd38f4b449 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:00.082017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe70f-5715-4550-960d-c6cd38f4b449 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:00.082760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbe70f-5715-4550-960d-c6cd38f4b449 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:00.101865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbe70f-5715-4550-960d-c6cd38f4b449 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:00.105275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbe70f-5715-4550-960d-c6cd38f4b449 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:00.136761 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1215/2406] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:40:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:00.744738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:00.747174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:00.800862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63e07bf-41ae-4782-a071-6acf0a6c7ec5 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:00.810642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:00.812068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:00.812792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:00.813847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] PATH_INFO: `/v3/projects/6e86de3db260403d88f0e8924463627c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:00.815430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e86de3db260403d88f0e8924463627c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:00.819134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:00.926427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e86de3db260403d88f0e8924463627c', 'request_id': 'req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:00.933158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e86de3db260403d88f0e8924463627c', 'request_id': 'req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:00.947931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e86de3db260403d88f0e8924463627c', 'request_id': 'req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:00.970570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] The token cache is being invalidate because project 6e86de3db260403d88f0e8924463627c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:41:00.970973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:00.972233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-1b73dea4-575c-4cdb-ab4d-9096bb223ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:00.977675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1192/2407] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:41:00 2022] DELETE /identity/v3/projects/6e86de3db260403d88f0e8924463627c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:00.994908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:00.994908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:00.994908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.008418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:01.009670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:01.031348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:01.092755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1216/2408] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:41:01.104926 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.105199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.105494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.147730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:01.149590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:01.159618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1193/2409] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:01.166371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.185929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afde27d-5860-424b-bdd6-4fe5aecc5ed8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.195546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.196574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.197058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.197542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.198375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:41:01.198774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.201167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.222016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1217/2410] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:41:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:41:01.227214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.242687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca31358-d447-4a89-b4d7-f436ba71b75b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.252738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.253488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.253808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.254219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.254998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.258414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.304802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1194/2411] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:41:01 2022] POST /identity/v3/projects => generated 389 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:41:01.309279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.320891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f321d76-4298-4c83-82ee-06a31a72d8af admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.415437 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.416410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.417064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.417185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.418299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.421054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.451224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1218/2412] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:41:01 2022] POST /identity/v3/users => generated 295 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:41:01.458686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.478471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2dcae3-5241-4ebb-b6bc-167eeac57830 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.483772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.484468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.484702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.484946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.486168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.488249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.504092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1195/2413] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:41:01.508968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.520785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7900e-740e-4ce4-85ae-1ec8a862bd2e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.526570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec000c1-a2c4-4a1f-924c-cbe53527a306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.527220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec000c1-a2c4-4a1f-924c-cbe53527a306 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.527505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec000c1-a2c4-4a1f-924c-cbe53527a306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.527810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec000c1-a2c4-4a1f-924c-cbe53527a306 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.528691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec000c1-a2c4-4a1f-924c-cbe53527a306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.531651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec000c1-a2c4-4a1f-924c-cbe53527a306 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.553024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1219/2414] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:41:01 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:41:01.557523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.578816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975388dd-1794-48ae-a2c1-f5b484db91c1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.583658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.584409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.584620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.584891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] PATH_INFO: `/v3/projects/3f16bc5d9aad416aa2973a17ef2da82b/users/7de8067a0b284ebaaadec5ff51b790f6/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.606224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f16bc5d9aad416aa2973a17ef2da82b, user_id=7de8067a0b284ebaaadec5ff51b790f6, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.609691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.625325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1196/2415] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:41:01 2022] PUT /identity/v3/projects/3f16bc5d9aad416aa2973a17ef2da82b/users/7de8067a0b284ebaaadec5ff51b790f6/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:01.647323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.648532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.649060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.692264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] MFA Rules not processed for user `7de8067a0b284ebaaadec5ff51b790f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:01.693669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:01.787495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1220/2416] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:01.799647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:01.801643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:01.827443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:01.875864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3e325-c364-4f7d-9526-9d7a4a4beeaf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:01.879671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:01.880301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:01.880578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:01.880850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:01.889982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:01.892059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:01.910523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1197/2417] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:01 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:05.008795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:05.010727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:05.031532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:05.072331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42953616-d12c-4cc0-93d6-fac184ee87d4 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:05.076651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:05.077633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:05.077633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:05.078517 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:05.080721 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:05.153230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:05.153820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:05.163909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1221/2418] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:11.898156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=17 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:15.159562 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=42 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:30.053759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:30.065161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241b08d2-e0cf-44b2-a3de-5845e546a259 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:30.068750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:30.069353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:30.069627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:30.069894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:30.071742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:30.176549 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:30.178465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:30.193108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1198/2419] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:31.453081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:31.455270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:31.467680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05128f9a-952c-4a94-ba93-4ab090c7e687 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:31.471461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:31.472053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:31.472331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:31.472600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:31.474320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:31.487585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:31.489007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c178febd-2479-4a05-857c-41236a8c637b service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:31.500644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1222/2420] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:35.136178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:35.220571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fb448f-2637-4c47-a38b-c275b99c617e service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:35.224192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:35.224829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:35.225105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:35.225402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] PATH_INFO: `/v3/users/7de8067a0b284ebaaadec5ff51b790f6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:35.226584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7de8067a0b284ebaaadec5ff51b790f6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:35.228552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:35.280515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7de8067a0b284ebaaadec5ff51b790f6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.281273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7de8067a0b284ebaaadec5ff51b790f6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.282055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7de8067a0b284ebaaadec5ff51b790f6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.289701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7de8067a0b284ebaaadec5ff51b790f6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.300523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7de8067a0b284ebaaadec5ff51b790f6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.323160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1199/2421] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:41:35 2022] DELETE /identity/v3/users/7de8067a0b284ebaaadec5ff51b790f6 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:35.327737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:35.328251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:35.328357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:35.347104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:35.348434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:35.413764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1223/2422] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:41:35.424469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:35.425967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:35.433557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:35.496110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27912c52-7f7b-4bbe-a711-4d2fe1f79a53 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:35.499875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c0469d-4f95-49cf-bf5f-9e1de5de5524 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:35.500479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0469d-4f95-49cf-bf5f-9e1de5de5524 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:35.501346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0469d-4f95-49cf-bf5f-9e1de5de5524 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:35.501627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c0469d-4f95-49cf-bf5f-9e1de5de5524 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:35.511066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0469d-4f95-49cf-bf5f-9e1de5de5524 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:35.512507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c0469d-4f95-49cf-bf5f-9e1de5de5524 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:35.523142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1200/2423] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:35.848202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:35.850081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:35.866736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d633c-40d9-4f2f-8733-3d18975b520e None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:35.873109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:35.874839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:35.875238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:35.876364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] PATH_INFO: `/v3/projects/3f16bc5d9aad416aa2973a17ef2da82b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:35.877975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f16bc5d9aad416aa2973a17ef2da82b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:35.880747 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:35.941729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f16bc5d9aad416aa2973a17ef2da82b', 'request_id': 'req-1729e91b-7711-4634-9739-bbafd377ff73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.948678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f16bc5d9aad416aa2973a17ef2da82b', 'request_id': 'req-1729e91b-7711-4634-9739-bbafd377ff73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.956167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f16bc5d9aad416aa2973a17ef2da82b', 'request_id': 'req-1729e91b-7711-4634-9739-bbafd377ff73'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.970261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] The token cache is being invalidate because project 3f16bc5d9aad416aa2973a17ef2da82b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:41:35.970740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.971723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1729e91b-7711-4634-9739-bbafd377ff73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:35.974220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1224/2424] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:41:35 2022] DELETE /identity/v3/projects/3f16bc5d9aad416aa2973a17ef2da82b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:35.979249 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:35.979530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:35.979807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.002076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:36.003078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:36.075540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1201/2425] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:36.084818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.085159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.085436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.115375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:36.115375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:36.120777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1225/2426] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:41:36.125062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.136032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f91ea9-db9a-4127-baf1-b6f59a5f1a94 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.140752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.141848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.142288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.142617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.143293 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:41:36.144010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.145879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.159557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1202/2427] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:41:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:41:36.163995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.174264 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a7b5b5-0c83-4062-84fd-d71a12c969df None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.179377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.180028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.180306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.180663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.181236 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.184192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.217886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1226/2428] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:41:36 2022] POST /identity/v3/projects => generated 397 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:41:36.225424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.241102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adcfe0e-3897-4adf-96a4-3335c18bdb45 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.324706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.325765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.326327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.326812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.328271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.332748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.368134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1203/2429] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:41:36 2022] POST /identity/v3/users => generated 299 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:41:36.369291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.383019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ee883-2dca-4d04-8ba7-e204415faa7f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.388852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.389620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.390455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.390455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.391640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.394513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.410020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1227/2430] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:41:36.422611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.435295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d297e1f8-f275-48b7-aaf8-ee535ea5983d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.442330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05afc40-6ec3-48fc-bfab-62670430a868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.443049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05afc40-6ec3-48fc-bfab-62670430a868 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.443412 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05afc40-6ec3-48fc-bfab-62670430a868 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.443747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05afc40-6ec3-48fc-bfab-62670430a868 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.444460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05afc40-6ec3-48fc-bfab-62670430a868 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.446143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05afc40-6ec3-48fc-bfab-62670430a868 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.460459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1204/2431] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:41:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:41:36.465133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.476523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93345af1-9bcb-47f0-97ad-607b787f8025 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.481678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.482388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.482707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.482948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] PATH_INFO: `/v3/projects/a6df7f74a35445f69bb03eb7eb8ad040/users/d29f1ae1e62b4ce79b8f3c219a559328/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.500050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6df7f74a35445f69bb03eb7eb8ad040, user_id=d29f1ae1e62b4ce79b8f3c219a559328, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.503417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.519958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1228/2432] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:41:36 2022] PUT /identity/v3/projects/a6df7f74a35445f69bb03eb7eb8ad040/users/d29f1ae1e62b4ce79b8f3c219a559328/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:36.604805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.605268 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.605617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.633958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:36.639774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] MFA Rules not processed for user `d29f1ae1e62b4ce79b8f3c219a559328`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:36.644492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:36.730967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1205/2433] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:36 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:36.743989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:36.746261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:36.756800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:36.810983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23103d-bfcc-4c0a-a6dd-a931032d9bba admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:36.815401 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:36.816022 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:36.816303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:36.816575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:36.826269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:36.827745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:36.839177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1229/2434] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:36 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:43.490843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:43.492241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:43.551453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1cca9-9544-4b01-964a-9fe13c55b110 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:43.555504 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:43.556113 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:43.556387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:43.556654 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:43.558547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:43.621491 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:43.623201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc858e3-1e03-4629-95fe-c63573cdf042 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:43.633484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1206/2435] 10.222.0.91 () {60 vars in 1396 bytes} [Wed Aug 24 03:41:43 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3264 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:46.837463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=23 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:47.170827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:47.243191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46cabf1-fa0e-456b-aae4-41526210d360 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:47.247802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:47.248383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.248662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.248934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] PATH_INFO: `/v3/users/d29f1ae1e62b4ce79b8f3c219a559328` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.250059 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d29f1ae1e62b4ce79b8f3c219a559328)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:47.252575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:47.309613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd29f1ae1e62b4ce79b8f3c219a559328'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.310415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd29f1ae1e62b4ce79b8f3c219a559328'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.311442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd29f1ae1e62b4ce79b8f3c219a559328'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.320762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd29f1ae1e62b4ce79b8f3c219a559328'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.337415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd29f1ae1e62b4ce79b8f3c219a559328'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.353129 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1230/2436] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:41:47 2022] DELETE /identity/v3/users/d29f1ae1e62b4ce79b8f3c219a559328 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:47.358940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.359397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.359684 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.388606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:47.389875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.446945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:47.459596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1207/2437] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:47.470497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:47.472024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.551755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd5352-9a58-4ba1-9f39-71374ba557f4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:47.555706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c8217a-6bb5-4cff-ab56-3b735abb5ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:47.556299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8217a-6bb5-4cff-ab56-3b735abb5ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.556577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8217a-6bb5-4cff-ab56-3b735abb5ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.556843 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c8217a-6bb5-4cff-ab56-3b735abb5ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.567117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8217a-6bb5-4cff-ab56-3b735abb5ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:47.568658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c8217a-6bb5-4cff-ab56-3b735abb5ef5 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:47.578315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1231/2438] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:47.664056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:47.666299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.680501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991b3828-7a65-4850-987a-30e6638fb03c None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:47.687659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:47.687659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.687863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.688141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] PATH_INFO: `/v3/users/ba24c2a2c8a849c0a3c8e19964017887` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.689327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba24c2a2c8a849c0a3c8e19964017887)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:47.691319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:47.762129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba24c2a2c8a849c0a3c8e19964017887'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.763218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba24c2a2c8a849c0a3c8e19964017887'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.764291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba24c2a2c8a849c0a3c8e19964017887'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.776721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba24c2a2c8a849c0a3c8e19964017887'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.789930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba24c2a2c8a849c0a3c8e19964017887'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:47.817539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1208/2439] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:41:47 2022] DELETE /identity/v3/users/ba24c2a2c8a849c0a3c8e19964017887 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:47.823188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.823605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.823971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.846004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:47.847309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.856919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:47.858475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.871637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:47.943967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c346f3-588d-4bcf-8e00-b17ac6be1449 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:47.952549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1232/2440] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:47.952961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:47.953451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:47.954468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.954859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.955225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.958892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.959588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:47.974299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85aa80d9-b849-4b06-9615-689e4d66d8f9 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:47.981357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:47.982012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:47.982382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:47.982724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] PATH_INFO: `/v3/projects/a6df7f74a35445f69bb03eb7eb8ad040` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:47.983885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6df7f74a35445f69bb03eb7eb8ad040)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:47.985847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:48.043693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:48.045669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:48.048337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6df7f74a35445f69bb03eb7eb8ad040', 'request_id': 'req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.054929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6df7f74a35445f69bb03eb7eb8ad040', 'request_id': 'req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.060780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1209/2441] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:41:48.061155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6df7f74a35445f69bb03eb7eb8ad040', 'request_id': 'req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.062210 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:48.065145 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:48.082923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] The token cache is being invalidate because project a6df7f74a35445f69bb03eb7eb8ad040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:41:48.083544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.084592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.087242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1233/2442] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:41:47 2022] DELETE /identity/v3/projects/a6df7f74a35445f69bb03eb7eb8ad040 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:48.119156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:48.168195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb9052b-1aa8-42af-8e7e-c0c276d914af service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:48.234843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:48.237152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:48.237689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:48.238444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:48.240494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:48.342310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:48.343868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb45d886-2453-4ee4-94fc-054e4bde05f7 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:48.355173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1210/2443] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:48.691147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:48.692810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:48.706436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc6dd6-b9b3-48a8-b04b-a543816f9051 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:48.711837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:48.712495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:48.712951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:48.713227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] PATH_INFO: `/v3/projects/8c0b7bcc238c4130a25594dc1becca2d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:48.714447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c0b7bcc238c4130a25594dc1becca2d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:48.716361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:48.769077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c0b7bcc238c4130a25594dc1becca2d', 'request_id': 'req-2e95d67e-82e0-4d23-acaa-9cc729da3460'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.775289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c0b7bcc238c4130a25594dc1becca2d', 'request_id': 'req-2e95d67e-82e0-4d23-acaa-9cc729da3460'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.781740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c0b7bcc238c4130a25594dc1becca2d', 'request_id': 'req-2e95d67e-82e0-4d23-acaa-9cc729da3460'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.795849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] The token cache is being invalidate because project 8c0b7bcc238c4130a25594dc1becca2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:41:48.796217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.796922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:41:48.799327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1234/2444] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:41:48 2022] DELETE /identity/v3/projects/8c0b7bcc238c4130a25594dc1becca2d => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:41:55.956578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:55.957129 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:55.957360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:55.977843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] MFA Rules not processed for user `b1d9d5fd0db54ddf85e05c77dd7f6c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:41:55.979049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:56.036001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1211/2445] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:41:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:41:56.355108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:56.356846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:56.391328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:56.419150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95d67e-82e0-4d23-acaa-9cc729da3460 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:56.423395 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:56.423964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:56.424238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:56.424505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:56.426225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:56.436144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:56.437558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab29a03a-8dc8-4719-93ed-b2ea2a4e268d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:56.446120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1235/2446] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:41:56.771563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:41:56.773022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:56.815921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:41:56.854140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f424ab-01a0-44de-bd6b-8dd11be57b12 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:41:56.859136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:41:56.859757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:41:56.860601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:41:56.860840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:41:56.862923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:41:56.875221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:41:56.877375 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:41:56.887753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1212/2447] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:41:56 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:00.250303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:00.251671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:00.252165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:00.284140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:00.286138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:00.377577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1236/2448] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:42:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:00.445108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:00.446728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:00.526076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406505ef-2beb-41be-972f-74dbbbde0c2c service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:00.535316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908d6ec3-0d49-4faa-8b1e-5d5363ba7816 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:00.536074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d6ec3-0d49-4faa-8b1e-5d5363ba7816 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:00.536227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d6ec3-0d49-4faa-8b1e-5d5363ba7816 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:00.536509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908d6ec3-0d49-4faa-8b1e-5d5363ba7816 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:00.546693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d6ec3-0d49-4faa-8b1e-5d5363ba7816 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:00.548966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908d6ec3-0d49-4faa-8b1e-5d5363ba7816 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:00.559416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1213/2449] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:42:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:42:08.455995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:08.473310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c4f54-5265-4aed-935d-40d7b7eefa51 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:08.481827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:08.482796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:08.483252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:08.483725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:08.486369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:08.506107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:08.507686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10371900-95c0-4e06-a761-daabe8c76f22 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:08.519271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1237/2450] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:10.555589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=48 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:16.411869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccdc5b-805e-4b10-8c97-d6930dc2d90a None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.412540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccdc5b-805e-4b10-8c97-d6930dc2d90a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.412540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfccdc5b-805e-4b10-8c97-d6930dc2d90a None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.413768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1214/2451] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:42:16.420571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c814-95a0-4aea-a205-6003ece2af7f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.421135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c814-95a0-4aea-a205-6003ece2af7f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.421135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241c814-95a0-4aea-a205-6003ece2af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.442753 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8241c814-95a0-4aea-a205-6003ece2af7f None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:16.443993 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8241c814-95a0-4aea-a205-6003ece2af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:16.556214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1238/2452] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:16.569155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccdc5b-805e-4b10-8c97-d6930dc2d90a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:16.570892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dfccdc5b-805e-4b10-8c97-d6930dc2d90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:16.636687 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfccdc5b-805e-4b10-8c97-d6930dc2d90a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:16.640315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254da73-529c-47ba-9ca3-7c44016affa2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:16.640919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254da73-529c-47ba-9ca3-7c44016affa2 service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.641211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254da73-529c-47ba-9ca3-7c44016affa2 service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.642803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254da73-529c-47ba-9ca3-7c44016affa2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.654816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254da73-529c-47ba-9ca3-7c44016affa2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:16.656273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f254da73-529c-47ba-9ca3-7c44016affa2 service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:16.667994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1215/2453] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:16.740947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed63a48-9f74-4d7c-baa2-be2f19a1262b None None] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.741221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed63a48-9f74-4d7c-baa2-be2f19a1262b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.741487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed63a48-9f74-4d7c-baa2-be2f19a1262b None None] PATH_INFO: `/` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.742879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1239/2454] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:42:16.749438 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13a359-d023-48bf-9f83-906d8a85b12a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.749768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13a359-d023-48bf-9f83-906d8a85b12a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.749990 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13a359-d023-48bf-9f83-906d8a85b12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.771344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ee13a359-d023-48bf-9f83-906d8a85b12a None None] MFA Rules not processed for user `5ae7930c17434ae1b6df030b0599e768`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:16.772702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ee13a359-d023-48bf-9f83-906d8a85b12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:16.780567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1216/2455] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:16.793159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed63a48-9f74-4d7c-baa2-be2f19a1262b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:16.805355 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed63a48-9f74-4d7c-baa2-be2f19a1262b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:16.809073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67475329-b816-49f6-8c52-5c7519e329ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:16.809676 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475329-b816-49f6-8c52-5c7519e329ec service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.809955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475329-b816-49f6-8c52-5c7519e329ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.810250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67475329-b816-49f6-8c52-5c7519e329ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.818000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67475329-b816-49f6-8c52-5c7519e329ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:16.819472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67475329-b816-49f6-8c52-5c7519e329ec service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:16.827998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1240/2456] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:16.918300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bc041-14bf-41ec-be11-2a9a38a7a791 None None] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.918583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bc041-14bf-41ec-be11-2a9a38a7a791 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.918862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60bc041-14bf-41ec-be11-2a9a38a7a791 None None] PATH_INFO: `/` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.920304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1217/2457] 10.222.0.91 () {54 vars in 921 bytes} [Wed Aug 24 03:42:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 24 03:42:16.926002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb82aec-8e6a-4deb-8160-5b2dc2fec7e4 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.926177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb82aec-8e6a-4deb-8160-5b2dc2fec7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.926422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb82aec-8e6a-4deb-8160-5b2dc2fec7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:16.947413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-beb82aec-8e6a-4deb-8160-5b2dc2fec7e4 None None] MFA Rules not processed for user `1d56ed96c2f546e59e473b1d62bf99eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:16.948619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-beb82aec-8e6a-4deb-8160-5b2dc2fec7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:16.958690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1241/2458] 10.222.0.91 () {58 vars in 1018 bytes} [Wed Aug 24 03:42:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:42:16.969387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bc041-14bf-41ec-be11-2a9a38a7a791 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:16.972250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e60bc041-14bf-41ec-be11-2a9a38a7a791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:16.986271 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60bc041-14bf-41ec-be11-2a9a38a7a791 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:16.989899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:16.990545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:16.990823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:16.991124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:17.006460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:17.007924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:17.019563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1218/2459] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:20.038894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:20.039443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:20.039443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:20.060805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:20.062315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:20.070307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1242/2460] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:42:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:20.086149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:20.100981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100a7f18-0392-4a56-ad49-d4683073f4ce service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:20.106899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f24fa3-a869-4d9f-9b41-c1e1fb37058b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:20.107510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24fa3-a869-4d9f-9b41-c1e1fb37058b service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:20.107793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24fa3-a869-4d9f-9b41-c1e1fb37058b service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:20.108067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f24fa3-a869-4d9f-9b41-c1e1fb37058b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:20.119886 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f24fa3-a869-4d9f-9b41-c1e1fb37058b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:20.121511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f24fa3-a869-4d9f-9b41-c1e1fb37058b service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:20.130761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1219/2461] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:42:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:42:30.054826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:30.127371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=41 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:42:50.534827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:50.536941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:50.554848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe48d98-5765-4bce-b645-22415893ea71 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:50.560770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:50.561393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:50.561670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:50.561948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:50.564656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:50.585895 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:50.587547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:50.606265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1243/2462] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:42:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:42:53.763921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:53.764500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:53.764729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:53.786230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:42:53.787472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:42:53.794740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1220/2463] 10.222.0.91 () {58 vars in 967 bytes} [Wed Aug 24 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:42:53.808664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:42:53.822552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d0de6-d5b2-4d9c-bf2f-edc78df29133 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:42:53.830403 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed2ebd-488a-4e80-a754-49d930ffbfd6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:42:53.831038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed2ebd-488a-4e80-a754-49d930ffbfd6 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:42:53.831314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed2ebd-488a-4e80-a754-49d930ffbfd6 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:42:53.831584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed2ebd-488a-4e80-a754-49d930ffbfd6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:42:53.840897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed2ebd-488a-4e80-a754-49d930ffbfd6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:42:53.842745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed2ebd-488a-4e80-a754-49d930ffbfd6 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:42:53.852165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1244/2464] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:42:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:43:03.780266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:03.848910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=12 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:16.864385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:16.866569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:16.887985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5752f4d2-145f-42bf-b94a-27c60f3d6d44 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:16.894102 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:16.894738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:16.895015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:16.895290 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:16.897023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:16.964371 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:16.966211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:16.976004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1221/2465] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:16 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:43:26.971486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=28 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:43:48.705524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:48.706119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:48.706119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:48.725534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] MFA Rules not processed for user `76c0df49fa9342778a56eae098b2f2cd`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:43:48.726825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:48.733771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1245/2466] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:43:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:43:55.325201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:43:55.339457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de03edb7-7187-4804-b8d1-79fe2e695bf4 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:43:55.348236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:43:55.348895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:43:55.349199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:43:55.349692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:43:55.351857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:43:55.365736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:43:55.367476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:43:55.378324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1222/2467] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:43:55 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:43:58.719968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:00.229423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:00.246331 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07e23c5-11f3-46b6-b09b-9ecd838978b3 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:00.253422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:00.254557 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:00.254901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:00.255239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:00.257163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:00.275362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:00.277108 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a285c-b0e1-464d-8c56-0316df1d8270 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:00.290107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1246/2468] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:00.502516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:00.513832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10d3957-2169-48a6-8b6e-8d25786da743 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:00.524229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:00.524845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:00.525114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:00.525584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:00.527759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:00.590988 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:00.592600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:00.603318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1223/2469] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:10.284521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:10.575417 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e0b19-3d9b-4dc4-bebd-ae299333f735 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:10.575707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e0b19-3d9b-4dc4-bebd-ae299333f735 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:10.575985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e0b19-3d9b-4dc4-bebd-ae299333f735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:10.598584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-bf7e0b19-3d9b-4dc4-bebd-ae299333f735 None None] MFA Rules not processed for user `b1d9d5fd0db54ddf85e05c77dd7f6c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:10.599253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=34 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:10.600369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bf7e0b19-3d9b-4dc4-bebd-ae299333f735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:10.607925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1247/2470] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:44:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:10.926887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:10.936423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c227cf91-ec03-491a-9aa4-f6c2a3998d97 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:10.940051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:10.940642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:10.940920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:10.941194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:10.943465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:10.955134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:10.956566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:10.965184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1224/2471] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:20.592783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:20.961531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:26.306641 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:26.307188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:26.307188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:26.337255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] MFA Rules not processed for user `0b294c0072b740cc960bc53e1f806ffc`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:26.338535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:26.424426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1248/2472] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:44:26 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:44:28.465772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:28.467871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:28.558142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d89b137-ffac-4b6c-9484-c2f731e547c1 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:28.564342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:28.564954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:28.565234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:28.565508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:28.567306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:28.580264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:28.582032 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783fc66-a663-4404-ba35-2a919dfc8b2d service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:28.595564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1225/2473] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:36.422295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=37 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:38.591653 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=31 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:44:50.471853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:50.473898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:50.541421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe37b399-3584-419b-a718-080ac7278d53 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:50.546017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:50.546681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:50.546964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:50.547238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:50.549039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:50.617802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:50.619273 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:50.631287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1249/2474] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:55.623867 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:55.624397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:55.624397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:55.663177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:44:55.665945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:55.684898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1226/2475] 10.222.0.91 () {58 vars in 967 bytes} [Wed Aug 24 03:44:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:44:55.743335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:55.756099 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9752d15-9c62-4bf4-ba66-f8510b36a4aa service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:55.770681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:55.771310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:55.771596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:55.771874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:55.781484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:55.783140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:55.792326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1250/2476] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:44:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:44:56.202087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:56.225356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121133f-f92f-48bc-87fd-5ae3efc33c44 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:56.229360 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:56.229974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:56.230285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:56.230554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:56.232373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:56.329257 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:56.331199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:56.354521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1227/2477] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:44:59.829566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:44:59.844161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1724f-a575-41d5-9b88-6ef9a1806fd8 service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:44:59.850005 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:44:59.850892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:44:59.851359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:44:59.851628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:44:59.853538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:44:59.870565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:44:59.871665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:44:59.886975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1251/2478] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:44:59 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:03.244388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:03.246866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:03.263098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d27e07a-6b20-492d-bdab-2cb67cfebe12 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:03.268157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:03.268734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.269013 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.269284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] PATH_INFO: `/v3/users/b9f929b986e645109db65651c8ff1502` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.271350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f929b986e645109db65651c8ff1502)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:03.273139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:03.325750 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f929b986e645109db65651c8ff1502'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.326479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f929b986e645109db65651c8ff1502'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.327122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f929b986e645109db65651c8ff1502'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.336887 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f929b986e645109db65651c8ff1502'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.359424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f929b986e645109db65651c8ff1502'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.399213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1228/2479] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:45:03 2022] DELETE /identity/v3/users/b9f929b986e645109db65651c8ff1502 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:45:03.607010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:03.621799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e975ac7-39e4-48c5-ab15-091acd0856cf service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:03.696734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:03.697449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.697698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.697968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] PATH_INFO: `/v3/projects/0fe6a4ca8c984b2aa4df4c47b4c1d03b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.699243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fe6a4ca8c984b2aa4df4c47b4c1d03b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:03.701191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:03.748492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:03.772388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fe6a4ca8c984b2aa4df4c47b4c1d03b', 'request_id': 'req-089f13de-331b-4bbb-87dc-e7a05ae01fa9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.778551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fe6a4ca8c984b2aa4df4c47b4c1d03b', 'request_id': 'req-089f13de-331b-4bbb-87dc-e7a05ae01fa9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.785042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fe6a4ca8c984b2aa4df4c47b4c1d03b', 'request_id': 'req-089f13de-331b-4bbb-87dc-e7a05ae01fa9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.799771 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] The token cache is being invalidate because project 0fe6a4ca8c984b2aa4df4c47b4c1d03b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:45:03.800058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.800733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.802786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1252/2480] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:45:03 2022] DELETE /identity/v3/projects/0fe6a4ca8c984b2aa4df4c47b4c1d03b => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:45:03.807041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:03.808517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:03.825021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a015fd-3802-457f-a455-d3cfa9162af6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:03.829088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:03.896213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:03.897528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:03.897808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:03.898735 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] PATH_INFO: `/v3/users/2a30c7b692d242869e99c96e9a04a567` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:03.899999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a30c7b692d242869e99c96e9a04a567)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:03.902042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:03.954685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a30c7b692d242869e99c96e9a04a567'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.955351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a30c7b692d242869e99c96e9a04a567'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.955957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a30c7b692d242869e99c96e9a04a567'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.964677 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a30c7b692d242869e99c96e9a04a567'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.973611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a30c7b692d242869e99c96e9a04a567'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:03.995676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1229/2481] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:45:03 2022] DELETE /identity/v3/users/2a30c7b692d242869e99c96e9a04a567 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:45:04.187067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:04.189283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:04.204476 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089f13de-331b-4bbb-87dc-e7a05ae01fa9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:04.272958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:04.273956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:04.274430 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:04.274906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] PATH_INFO: `/v3/projects/42ba9acaf363498ea12cedff000a6005` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:04.276731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ba9acaf363498ea12cedff000a6005)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:04.279968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:04.349393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ba9acaf363498ea12cedff000a6005', 'request_id': 'req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:04.356743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ba9acaf363498ea12cedff000a6005', 'request_id': 'req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:04.364652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ba9acaf363498ea12cedff000a6005', 'request_id': 'req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:04.378275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] The token cache is being invalidate because project 42ba9acaf363498ea12cedff000a6005 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:45:04.378565 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:04.379797 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:04.381307 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1253/2482] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:45:04 2022] DELETE /identity/v3/projects/42ba9acaf363498ea12cedff000a6005 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:45:13.991199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=50,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:14.374473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=64,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:29.079023 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:29.080554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:29.167081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110faf2-ae9a-4241-88cb-7f7f770d57a4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:29.223864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:29.225012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:29.225301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:29.225578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:29.227490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:29.294254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:29.295757 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:29.305692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1230/2483] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:33.288786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:33.290298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:33.304677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2f06ee-d333-449c-9a66-9aa5bdbf0ac8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:33.309646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:33.310351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.310632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.310906 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] PATH_INFO: `/v3/users/76c0df49fa9342778a56eae098b2f2cd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.312350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76c0df49fa9342778a56eae098b2f2cd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:33.314350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:33.373464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76c0df49fa9342778a56eae098b2f2cd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.374500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76c0df49fa9342778a56eae098b2f2cd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.375262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76c0df49fa9342778a56eae098b2f2cd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.386439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76c0df49fa9342778a56eae098b2f2cd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.400364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-02d627f9-51f6-49ae-abe8-b1c734c641f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76c0df49fa9342778a56eae098b2f2cd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.414813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1254/2484] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:45:33 2022] DELETE /identity/v3/users/76c0df49fa9342778a56eae098b2f2cd => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:45:33.621625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:33.632827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b0e8cc-3722-4b71-a247-449060947db0 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:33.694442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:33.702748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:33.703467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.703916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.704204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] PATH_INFO: `/v3/projects/9494dfc650374d64bf4b5f46259a0171` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.705515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] RBAC: Authorizing `identity:delete_project(project_id=9494dfc650374d64bf4b5f46259a0171)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:33.707936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:33.768661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9494dfc650374d64bf4b5f46259a0171', 'request_id': 'req-308f4628-7f81-4df4-8d73-7b845d76face'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.775927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9494dfc650374d64bf4b5f46259a0171', 'request_id': 'req-308f4628-7f81-4df4-8d73-7b845d76face'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.785142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9494dfc650374d64bf4b5f46259a0171', 'request_id': 'req-308f4628-7f81-4df4-8d73-7b845d76face'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.797073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] The token cache is being invalidate because project 9494dfc650374d64bf4b5f46259a0171 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:45:33.797367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.798154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-308f4628-7f81-4df4-8d73-7b845d76face admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:45:33.800223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1231/2485] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:45:33 2022] DELETE /identity/v3/projects/9494dfc650374d64bf4b5f46259a0171 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:45:33.805809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.806093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.806413 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.866079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:33.867326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:33.931078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1255/2486] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:33.940373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.940723 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.940919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.961765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:33.963026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:33.971207 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1232/2487] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:45:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:33.975475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:33.985722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c698c66-dfae-4cf2-b389-f74202812585 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:33.991179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:33.991782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:33.992064 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:33.992336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:33.992980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:45:33.993282 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:33.994925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.008733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1256/2488] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:45:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:45:34.014066 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.029761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5478dcb4-a15a-4eb5-b1e8-c006790a6c81 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.034766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.035448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.035717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.035993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.036637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.039096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.068136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1233/2489] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:45:34 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:45:34.072529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.083310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0c2b9d-884a-4b79-a95c-0b57549b0e71 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.112254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:34.147897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.148585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.148871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.149149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.149782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.151982 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.184301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1257/2490] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:45:34 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:45:34.188283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.199641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97502a43-2278-4850-b4c1-3a2e01a17918 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.204552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.205162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.205442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.205714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.206556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.208174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.218895 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1234/2491] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:45:34.223101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.234083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c9ce0-7c6d-4932-838a-11c1a165a847 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.238984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a768d5-bf60-4e78-9197-56dea1fddbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.239626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a768d5-bf60-4e78-9197-56dea1fddbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.239921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a768d5-bf60-4e78-9197-56dea1fddbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.240295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a768d5-bf60-4e78-9197-56dea1fddbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.241446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a768d5-bf60-4e78-9197-56dea1fddbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.243121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a768d5-bf60-4e78-9197-56dea1fddbd3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.254682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1258/2492] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:45:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:45:34.259951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.270992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82745fe5-c4ba-475f-8b9d-d1ba4cd85f32 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.276746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.277404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.277680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.277946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] PATH_INFO: `/v3/projects/28b19b24989f449094001356712fc0e0/users/fe0dc1ce495045e4bf158ec503a5b5d1/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.293258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=28b19b24989f449094001356712fc0e0, user_id=fe0dc1ce495045e4bf158ec503a5b5d1, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.295958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.310424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1235/2493] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:45:34 2022] PUT /identity/v3/projects/28b19b24989f449094001356712fc0e0/users/fe0dc1ce495045e4bf158ec503a5b5d1/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:45:34.315977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ac43a-6d77-4072-a1f8-b2a176c88369 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.316473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ac43a-6d77-4072-a1f8-b2a176c88369 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.316619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022ac43a-6d77-4072-a1f8-b2a176c88369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.336707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-022ac43a-6d77-4072-a1f8-b2a176c88369 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:34.337926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-022ac43a-6d77-4072-a1f8-b2a176c88369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:34.398691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1259/2494] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:45:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:45:34.409503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:34.412122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:34.475136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddaa8a-94b0-4036-a2f7-6035db73dd76 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:34.483135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:34.483748 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:34.484022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:34.484285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:34.494743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:34.496346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:34.509897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1236/2495] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:38.808729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc44bd-eddf-4241-b4a5-5c55ad127805 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:38.809254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc44bd-eddf-4241-b4a5-5c55ad127805 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:38.809254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc44bd-eddf-4241-b4a5-5c55ad127805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:38.845467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0efc44bd-eddf-4241-b4a5-5c55ad127805 None None] MFA Rules not processed for user `fe0dc1ce495045e4bf158ec503a5b5d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:38.847010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0efc44bd-eddf-4241-b4a5-5c55ad127805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:38.934593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1260/2496] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:45:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:38.956162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:45:38.958423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:39.008156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:39.093312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9705da8c-4f1c-4470-8e10-a329fdd4d5b0 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:45:39.100351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c9154-6785-4ab9-944b-083eb025ec28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:45:39.100971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c9154-6785-4ab9-944b-083eb025ec28 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:39.101247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c9154-6785-4ab9-944b-083eb025ec28 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:39.101515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c9154-6785-4ab9-944b-083eb025ec28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:39.117155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c9154-6785-4ab9-944b-083eb025ec28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:45:39.118601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c9154-6785-4ab9-944b-083eb025ec28 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:45:39.128447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1237/2497] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:45:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:45:40.713354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:45:40.713855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:45:40.713996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:45:40.736678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] MFA Rules not processed for user `fe0dc1ce495045e4bf158ec503a5b5d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:45:40.738381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:45:40.747245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1261/2498] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:45:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:45:49.124019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=32 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:45:50.728796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:28.559692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:28.560260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:28.560260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:28.600918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] MFA Rules not processed for user `b1d9d5fd0db54ddf85e05c77dd7f6c89`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:28.602213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:28.661044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1238/2499] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:46:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:29.097754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:29.108535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d071736-080d-4d0c-bb33-13874065dd9c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:29.112206 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:29.112929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:29.113213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:29.113485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:29.115255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:29.129938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:29.131410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef839809-3af8-4038-8d14-d0d4957f6093 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:29.144195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1262/2500] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:31.536778 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:31.538557 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:31.552067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3687d3a6-b8be-477e-b9b7-b813a94ed855 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:31.567019 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:31.567755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:31.568124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:31.568463 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:31.570425 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:31.586602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:31.588242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:31.601306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1239/2501] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:38.639350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:38.639891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:38.639891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:38.667232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:38.668649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:38.754372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1263/2502] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:46:38 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:38.822298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:38.824337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:38.911217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a813b1ae-8a70-4bc3-b616-4cfad4dc8688 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:38.917306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:38.917933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:38.918236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:38.918513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:38.927906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:38.929536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:38.938780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1240/2503] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:46:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:46:43.179215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:43.195271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8daf654-2530-4f31-aa5b-5f8cf620d578 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:43.202575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:43.203426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:43.203877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:43.204226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:43.206119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:43.438619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:43.440520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:43.505950 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1264/2504] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:43 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:43.722804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:43.733873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3f794d-003b-484c-ba36-f2e24ebaa03b service swift] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:43.738576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:43.739400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:43.739842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:43.740286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:43.742744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:43.760645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:43.763155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:43.777584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1241/2505] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:46:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:46:51.718094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:51.719610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:51.732392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=73 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:51.751530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef5e298-61cd-4289-b7b1-6f390a839e13 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:51.756276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:51.757047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:51.757252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:51.757521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] PATH_INFO: `/v3/users/c0c89aa8c55042e7af5452f1a94318a9` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:51.758785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0c89aa8c55042e7af5452f1a94318a9)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:51.760783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:51.830951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0c89aa8c55042e7af5452f1a94318a9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:51.832057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0c89aa8c55042e7af5452f1a94318a9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:51.832866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0c89aa8c55042e7af5452f1a94318a9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:51.844911 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0c89aa8c55042e7af5452f1a94318a9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:51.868280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0c89aa8c55042e7af5452f1a94318a9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:51.884863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1265/2506] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:46:51 2022] DELETE /identity/v3/users/c0c89aa8c55042e7af5452f1a94318a9 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:46:52.355063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:52.360319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=79 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:52.366153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6a74d-b4ba-4bea-9b83-da605c9b58fb service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:52.433521 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:52.434159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:52.434468 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:52.434744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] PATH_INFO: `/v3/projects/cc4ed4d70a074b97acf12130e4b00bcb` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:52.435922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc4ed4d70a074b97acf12130e4b00bcb)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:52.437850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:52.490310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc4ed4d70a074b97acf12130e4b00bcb', 'request_id': 'req-edee04d3-ea45-4a13-9f40-86b1b02fc333'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.497173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc4ed4d70a074b97acf12130e4b00bcb', 'request_id': 'req-edee04d3-ea45-4a13-9f40-86b1b02fc333'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.505609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc4ed4d70a074b97acf12130e4b00bcb', 'request_id': 'req-edee04d3-ea45-4a13-9f40-86b1b02fc333'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.518984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] The token cache is being invalidate because project cc4ed4d70a074b97acf12130e4b00bcb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:46:52.519266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.519932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.521407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1242/2507] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:46:52 2022] DELETE /identity/v3/projects/cc4ed4d70a074b97acf12130e4b00bcb => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:46:52.526327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:52.527775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:52.540770 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3efb4f2-b2e2-4871-bad5-7e863dd255e4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:52.610602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:52.611146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:52.611405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:52.611680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] PATH_INFO: `/v3/users/bdf8485a95454f7d80eb4b31656006fd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:52.612858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf8485a95454f7d80eb4b31656006fd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:52.614849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:52.668257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf8485a95454f7d80eb4b31656006fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.668942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf8485a95454f7d80eb4b31656006fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.669563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf8485a95454f7d80eb4b31656006fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.678485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf8485a95454f7d80eb4b31656006fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.697933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da0a7eaa-d2ec-4e8d-8386-f416f93916bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf8485a95454f7d80eb4b31656006fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:52.702653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:52.719139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1266/2508] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:46:52 2022] DELETE /identity/v3/users/bdf8485a95454f7d80eb4b31656006fd => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:46:52.933870 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:52.936256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:52.955315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edee04d3-ea45-4a13-9f40-86b1b02fc333 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:53.025058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:53.026030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:53.029217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:53.032368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] PATH_INFO: `/v3/projects/318246d199744f0681068cf0310d0f67` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:53.034303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=318246d199744f0681068cf0310d0f67)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:53.036286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:53.085021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:46:53.101599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '318246d199744f0681068cf0310d0f67', 'request_id': 'req-9236f69b-575b-4f2b-b209-c650ef6fb477'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:53.108710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '318246d199744f0681068cf0310d0f67', 'request_id': 'req-9236f69b-575b-4f2b-b209-c650ef6fb477'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:53.121626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '318246d199744f0681068cf0310d0f67', 'request_id': 'req-9236f69b-575b-4f2b-b209-c650ef6fb477'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:53.137435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] The token cache is being invalidate because project 318246d199744f0681068cf0310d0f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:46:53.137732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:53.138469 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:46:53.140133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1243/2509] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:46:52 2022] DELETE /identity/v3/projects/318246d199744f0681068cf0310d0f67 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:46:57.315679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:57.316161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:57.316291 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:57.335057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:46:57.336196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:57.392766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1267/2510] 10.222.0.91 () {58 vars in 1009 bytes} [Wed Aug 24 03:46:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:46:57.404547 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:46:57.405930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:46:57.474942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9236f69b-575b-4f2b-b209-c650ef6fb477 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:46:57.479998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b497fb-8105-4b30-8399-daf95eca1de3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:46:57.480598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b497fb-8105-4b30-8399-daf95eca1de3 service swift] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:46:57.480880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b497fb-8105-4b30-8399-daf95eca1de3 service swift] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:46:57.481157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b497fb-8105-4b30-8399-daf95eca1de3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:46:57.489273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b497fb-8105-4b30-8399-daf95eca1de3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:46:57.490913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b497fb-8105-4b30-8399-daf95eca1de3 service swift] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:46:57.499570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1244/2511] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:46:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 24 03:47:07.386014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=28 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:47:07.496287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=5,keystone:SELECT=43,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:47:16.794632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:47:16.796096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:16.857540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f041ca06-f82d-48fd-b24d-bbc5aa3a0fd6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:47:16.861422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:47:16.862035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:47:16.862353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:47:16.862624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:47:16.864384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:16.874393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:47:16.875830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:47:16.883917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1268/2512] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:47:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:47:18.610267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:47:18.610955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:47:18.610955 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:47:18.675553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] MFA Rules not processed for user `787f0e0c0d9d42c999c77c7eb75d4ceb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:47:18.676789 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:47:18.694678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1245/2513] 10.222.0.91 () {58 vars in 967 bytes} [Wed Aug 24 03:47:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:47:18.708978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:47:18.732830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ffe442-6750-45c0-bf90-7232418f0955 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:47:18.737948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c97977ebdd6f4f6da887c8a7215bda69', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:47:18.738584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:47:18.738864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:47:18.739136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:47:18.748301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:47:18.749886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:47:18.758807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1269/2514] 10.222.0.91 () {60 vars in 1384 bytes} [Wed Aug 24 03:47:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 24 03:47:28.691764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=18 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:47:28.755297 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=36 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:13.297986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:13.299488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:13.371479 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278d47ce-f8a2-4493-9ac6-bab3ccaa51c3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:13.376880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:13.377482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:13.377762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:13.378035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:13.379813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:13.438629 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:13.440449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:13.459400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1246/2515] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:13 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:23.456662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=60 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:34.505919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:34.507408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:34.519267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e2da0f-1610-40f7-bc86-a4d62a19cfbf service swift] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:34.522867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:34.523471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:34.523739 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:34.524006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:34.525752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:34.590810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:34.592250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:34.602786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1270/2516] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:34.708972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:34.719178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc8d89-4588-4c51-b513-d82b75e8e148 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:34.722879 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:34.723480 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:34.723762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:34.724036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:34.726283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:34.738643 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:34.740091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:34.750432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1247/2517] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:39.107442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:39.107442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.125899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:39.127388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.128232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1147c31b-cfcd-4778-9093-8bce2ac613f4 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:39.134518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:39.135698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.135990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.136466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] PATH_INFO: `/v3/users/b1d9d5fd0db54ddf85e05c77dd7f6c89` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.137834 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d9d5fd0db54ddf85e05c77dd7f6c89)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:39.139894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:39.252415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d9d5fd0db54ddf85e05c77dd7f6c89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.253283 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d9d5fd0db54ddf85e05c77dd7f6c89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.253880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700a122-00b4-41eb-b447-8563b4f9138d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:39.259015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d9d5fd0db54ddf85e05c77dd7f6c89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.264123 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:39.264771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.265197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.265615 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.267638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.269017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d9d5fd0db54ddf85e05c77dd7f6c89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.283262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d9d5fd0db54ddf85e05c77dd7f6c89'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.286379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:39.288627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:39.296328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1271/2518] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:39 2022] DELETE /identity/v3/users/b1d9d5fd0db54ddf85e05c77dd7f6c89 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:39.299872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1248/2519] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:39.300803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:39.302173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.386907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf9bc3e-c822-45bd-be69-22b0b81511f9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:39.392185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:39.392549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.392833 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.393110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] PATH_INFO: `/v3/users/0b294c0072b740cc960bc53e1f806ffc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.394337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b294c0072b740cc960bc53e1f806ffc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:39.396320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:39.402810 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:39.452326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b294c0072b740cc960bc53e1f806ffc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.453017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b294c0072b740cc960bc53e1f806ffc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.453645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b294c0072b740cc960bc53e1f806ffc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.468646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b294c0072b740cc960bc53e1f806ffc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.483635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b294c0072b740cc960bc53e1f806ffc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.508167 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1272/2520] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:39 2022] DELETE /identity/v3/users/0b294c0072b740cc960bc53e1f806ffc => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:39.516702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:39.518231 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.553754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb522fc6-015f-46b5-8fb0-89066c2c2a62 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:39.657171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:39.657885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.657885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.658095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] PATH_INFO: `/v3/projects/c7c94d613f6b440bae60fc442661c671` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.659380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c94d613f6b440bae60fc442661c671)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:39.661377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:39.719554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:39.721553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.736534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c94d613f6b440bae60fc442661c671', 'request_id': 'req-dfc181e7-31f6-4653-b204-9930f862c1a6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.742648 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90afee5-5e4a-4b8e-8b29-87806b5e0739 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:39.743660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c94d613f6b440bae60fc442661c671', 'request_id': 'req-dfc181e7-31f6-4653-b204-9930f862c1a6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.752808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c94d613f6b440bae60fc442661c671', 'request_id': 'req-dfc181e7-31f6-4653-b204-9930f862c1a6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.766532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] The token cache is being invalidate because project c7c94d613f6b440bae60fc442661c671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:39.766820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.767460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-dfc181e7-31f6-4653-b204-9930f862c1a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.769008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1249/2521] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:39 2022] DELETE /identity/v3/projects/c7c94d613f6b440bae60fc442661c671 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:39.773958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc269472-cee6-4c69-99b4-abc7ea5e32dc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.783051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc269472-cee6-4c69-99b4-abc7ea5e32dc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.783051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc269472-cee6-4c69-99b4-abc7ea5e32dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.790525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:39.796632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cc269472-cee6-4c69-99b4-abc7ea5e32dc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:39.799049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cc269472-cee6-4c69-99b4-abc7ea5e32dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.815683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:39.816354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.816744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.817026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] PATH_INFO: `/v3/projects/6f0007a8402e4889bc8ed99a507036e4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.818205 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f0007a8402e4889bc8ed99a507036e4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:39.820443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:39.863282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1250/2522] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:39.868082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f0007a8402e4889bc8ed99a507036e4', 'request_id': 'req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.871605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.871892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.872160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.873961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f0007a8402e4889bc8ed99a507036e4', 'request_id': 'req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.880435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f0007a8402e4889bc8ed99a507036e4', 'request_id': 'req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.892984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:39.893502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] The token cache is being invalidate because project 6f0007a8402e4889bc8ed99a507036e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:39.893796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.894282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.894447 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-252256f0-bd4e-4cdd-8f93-3c88e001a7e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:39.896060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1273/2523] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:39 2022] DELETE /identity/v3/projects/6f0007a8402e4889bc8ed99a507036e4 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:39.901294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83608eb-dbd3-4b8d-ba83-914c78e2d2b9 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.901531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83608eb-dbd3-4b8d-ba83-914c78e2d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.901839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83608eb-dbd3-4b8d-ba83-914c78e2d2b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.920971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b83608eb-dbd3-4b8d-ba83-914c78e2d2b9 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:39.922215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b83608eb-dbd3-4b8d-ba83-914c78e2d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:39.931736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:39.956664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1251/2524] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:39.961400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:39.970531 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4246dabe-aed8-4e69-a281-bc488ffccc74 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:39.975627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:39.976239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.976495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.976765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.977422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:39.977708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:39.979344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:39.984086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1274/2525] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:39.994587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1252/2526] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:48:39.994809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:39.994809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:39.994809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:39.995351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.005152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c55e6ff-0037-4476-8ed4-abbe69fbe632 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.010917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.011526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.011807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.012083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.012361 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:40.012713 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.013581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:40.015446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.023881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1275/2527] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:40.028068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.037009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94041fa-dc83-43d8-b869-0be3e077413f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.041786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.042496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.042688 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.042965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.043619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:40.043924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.045542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.058044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1276/2528] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:48:40.062433 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.071779 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1253/2529] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:39 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:40.072818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76401d5-c21a-420e-ba03-44ec920cb122 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.072959 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.135811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.136424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.136698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.136966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.137584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.139524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.140620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413bbce-5acc-420a-915c-a4220a857f70 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.145571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.146177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.146490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.146764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.147388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.149320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.159543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:40.172202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1277/2530] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:40.173233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.174240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1254/2531] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:40.175581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.185666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d440374-05bf-4843-9eb9-32f489bf0663 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.238105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1567f4b-cdb8-491b-9a6e-80ea4a89d9da admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.243269 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.243878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.244119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.244289 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.244418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.244727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.245003 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.245112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.245274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.245914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.246998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.247548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.261475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1255/2532] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:48:40.262588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.272513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15406350-3d89-4176-9f68-398b29aefdb6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.276009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1278/2533] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:40.277119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.277348 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.277714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.277994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.278296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.278950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.280550 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.286967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a78f03-7cea-477a-9ff8-53a0beb70caa admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.291712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1256/2534] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:48:40.291877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.292333 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.292616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.292887 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.293523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.295198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.295763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.301109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:40.306437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee15f66-fac1-499f-bb8d-415bd6ea7226 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.308251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1279/2535] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:48:40.311037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.311964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73d3de-9159-4855-9494-61c8ceee7c40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.312632 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73d3de-9159-4855-9494-61c8ceee7c40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.312951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73d3de-9159-4855-9494-61c8ceee7c40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.313304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73d3de-9159-4855-9494-61c8ceee7c40 admin admin] PATH_INFO: `/v3/projects/38912b0899834adb8988b455bdfcfd27/users/85d486bc02cb460e8007ec67b5cdb0f2/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.323150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb44daa-754c-4e02-931b-db1d1ca47e03 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.328367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.328971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.329253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.329529 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.330163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.330353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73d3de-9159-4855-9494-61c8ceee7c40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38912b0899834adb8988b455bdfcfd27, user_id=85d486bc02cb460e8007ec67b5cdb0f2, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.331818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.333101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73d3de-9159-4855-9494-61c8ceee7c40 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.342518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1280/2536] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:48:40.346857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.350076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1257/2537] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:40 2022] PUT /identity/v3/projects/38912b0899834adb8988b455bdfcfd27/users/85d486bc02cb460e8007ec67b5cdb0f2/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:40.359954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.360248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c7974-ed69-4e6b-97b0-25fb2f4447b6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.360372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.360507 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.389049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] MFA Rules not processed for user `85d486bc02cb460e8007ec67b5cdb0f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:40.390319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:40.417837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f38b83-71d4-41d9-b428-9ca7ccc3e1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.418541 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f38b83-71d4-41d9-b428-9ca7ccc3e1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.418822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f38b83-71d4-41d9-b428-9ca7ccc3e1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.419123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f38b83-71d4-41d9-b428-9ca7ccc3e1c9 admin admin] PATH_INFO: `/v3/projects/5a1b1ecf1645424e9d5fa215563ae105/users/001473688de5486eacd1e97ac614402a/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.434500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f38b83-71d4-41d9-b428-9ca7ccc3e1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1b1ecf1645424e9d5fa215563ae105, user_id=001473688de5486eacd1e97ac614402a, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.437286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f38b83-71d4-41d9-b428-9ca7ccc3e1c9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.452077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1281/2538] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:40 2022] PUT /identity/v3/projects/5a1b1ecf1645424e9d5fa215563ae105/users/001473688de5486eacd1e97ac614402a/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:48:40.462286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.462573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.462966 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.463093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1258/2539] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:40.473699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.475720 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:40.491752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] MFA Rules not processed for user `001473688de5486eacd1e97ac614402a`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:40.492915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:40.525693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:40.558866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1282/2540] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:40.559216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dec1d4-05b3-440d-abba-0a895e627518 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.564548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.630464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.631792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.632096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.632428 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.634971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40f1947-9dc0-4005-9366-69437a9db1c6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.642426 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.643075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.643457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.643728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.644357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.646388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.653275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:40.687104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1283/2541] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:40.688423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.703218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.705035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.715546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1259/2542] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:48:40.784381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e12f49-eace-4f1f-80d2-b93d4f91e981 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.789643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.790303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.790586 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.790869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.791502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.793465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.817047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1284/2543] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:40 2022] POST /identity/v3/users => generated 293 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:40.822891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.834076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1314f476-6a1f-432c-bbea-ba47e7de1bb4 service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.842649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.843260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.843536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.843802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.844449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.846070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.859856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1260/2544] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:40.861116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.872118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47843c2-12a9-4d40-a541-0306bfcca750 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:40.877366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:40.877975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:40.878290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:40.878560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] PATH_INFO: `/v3/projects/de3c55f8863e4409872ea44599b77c0c/users/05b6c58be2224cc98695e06393b01e44/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:40.897330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3c55f8863e4409872ea44599b77c0c, user_id=05b6c58be2224cc98695e06393b01e44, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:40.900696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:40.915610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1285/2545] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:40 2022] PUT /identity/v3/projects/de3c55f8863e4409872ea44599b77c0c/users/05b6c58be2224cc98695e06393b01e44/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:40.920046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:40.934001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982ecd56-67d8-4077-b532-7937fd4e05ac admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:41.000230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2ca25c-f5b5-4697-baf2-5bd3e92cc567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:41.001138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca25c-f5b5-4697-baf2-5bd3e92cc567 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:41.001424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca25c-f5b5-4697-baf2-5bd3e92cc567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:41.001673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2ca25c-f5b5-4697-baf2-5bd3e92cc567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:41.002416 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca25c-f5b5-4697-baf2-5bd3e92cc567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:41.004364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2ca25c-f5b5-4697-baf2-5bd3e92cc567 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:41.017417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1261/2546] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:48:41.022262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:41.033385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed2fb28-5c25-4c04-8c4b-225c490bd880 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:41.040362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:41.041043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:41.041327 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:41.041651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] PATH_INFO: `/v3/projects/de3c55f8863e4409872ea44599b77c0c/users/05b6c58be2224cc98695e06393b01e44/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:41.043790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3c55f8863e4409872ea44599b77c0c, user_id=05b6c58be2224cc98695e06393b01e44, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:41.046614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:41.060171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1286/2547] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:41 2022] PUT /identity/v3/projects/de3c55f8863e4409872ea44599b77c0c/users/05b6c58be2224cc98695e06393b01e44/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:41.072773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:41.073062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:41.073336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:41.086540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:41.119404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] MFA Rules not processed for user `05b6c58be2224cc98695e06393b01e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:41.121039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:41.205355 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1262/2548] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:41 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:41.216967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:41.218905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:41.305472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceec7ac-ab90-4d61-a539-1e7dfd6980de admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:41.309366 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2990b-4840-479e-afa9-32a37d411677 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:41.310044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2990b-4840-479e-afa9-32a37d411677 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:41.310379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2990b-4840-479e-afa9-32a37d411677 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:41.310661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2990b-4840-479e-afa9-32a37d411677 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:41.320669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2990b-4840-479e-afa9-32a37d411677 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:41.322116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2990b-4840-479e-afa9-32a37d411677 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:41.332178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1287/2549] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:41 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:41.742772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:41.807878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee8aba-dc8c-4a94-a363-0afa335c9a4d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:41.812791 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:41.813445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:41.813746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:41.814021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] PATH_INFO: `/v3/users/001473688de5486eacd1e97ac614402a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:41.815534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=001473688de5486eacd1e97ac614402a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:41.818332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:41.870348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001473688de5486eacd1e97ac614402a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:41.871017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001473688de5486eacd1e97ac614402a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:41.871624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001473688de5486eacd1e97ac614402a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:41.879065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001473688de5486eacd1e97ac614402a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:41.887435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001473688de5486eacd1e97ac614402a'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:41.910140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1263/2550] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:41 2022] DELETE /identity/v3/users/001473688de5486eacd1e97ac614402a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:41.915769 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:41.916055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:41.916335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:41.938873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:41.940075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:41.958570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:42.005354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1288/2551] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:42.015380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:42.017587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:42.036865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:42.087511 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b8ffaa-345f-4e41-83d9-a00d20a26ff1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:42.091291 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:42.091900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:42.092180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:42.092449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:42.102239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:42.103696 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:42.113737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1264/2552] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:42.407018 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:42.408830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:42.422894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132db306-d206-4184-90b5-50b4ea3e8a69 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:42.431011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:42.431749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:42.432041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:42.432479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] PATH_INFO: `/v3/projects/5a1b1ecf1645424e9d5fa215563ae105` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:42.433809 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1b1ecf1645424e9d5fa215563ae105)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:42.435768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:42.486539 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1b1ecf1645424e9d5fa215563ae105', 'request_id': 'req-85686c80-1551-4dc0-8173-10ed783ff1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.493052 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1b1ecf1645424e9d5fa215563ae105', 'request_id': 'req-85686c80-1551-4dc0-8173-10ed783ff1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.499696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1b1ecf1645424e9d5fa215563ae105', 'request_id': 'req-85686c80-1551-4dc0-8173-10ed783ff1b1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.514376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] The token cache is being invalidate because project 5a1b1ecf1645424e9d5fa215563ae105 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:42.514376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.514626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.515962 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1289/2553] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:42 2022] DELETE /identity/v3/projects/5a1b1ecf1645424e9d5fa215563ae105 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:42.520152 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:42.532705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0296388b-0f6b-40fc-b38b-4a6722a476c4 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:42.590454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:42.590999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:42.591281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:42.591553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] PATH_INFO: `/v3/users/05b6c58be2224cc98695e06393b01e44` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:42.592676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=05b6c58be2224cc98695e06393b01e44)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:42.594658 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:42.639348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05b6c58be2224cc98695e06393b01e44'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.640000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05b6c58be2224cc98695e06393b01e44'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.640603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05b6c58be2224cc98695e06393b01e44'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.647604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05b6c58be2224cc98695e06393b01e44'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.655015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-8beb38d9-9cf5-4a8e-be6e-d3f322626f7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05b6c58be2224cc98695e06393b01e44'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:42.671737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1265/2554] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:42 2022] DELETE /identity/v3/users/05b6c58be2224cc98695e06393b01e44 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:48:42.943402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:42.945525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:42.961756 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85686c80-1551-4dc0-8173-10ed783ff1b1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.030632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.032042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.033372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.033634 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] PATH_INFO: `/v3/projects/de3c55f8863e4409872ea44599b77c0c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.034900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3c55f8863e4409872ea44599b77c0c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.036825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.086558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:43.088071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3c55f8863e4409872ea44599b77c0c', 'request_id': 'req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.096335 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3c55f8863e4409872ea44599b77c0c', 'request_id': 'req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.103150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3c55f8863e4409872ea44599b77c0c', 'request_id': 'req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.119026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] The token cache is being invalidate because project de3c55f8863e4409872ea44599b77c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:43.119233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.120343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1167b3c-d184-4e13-ba08-5e545c5ec2b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.121486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1290/2555] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:42 2022] DELETE /identity/v3/projects/de3c55f8863e4409872ea44599b77c0c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:48:43.127158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.127442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.127711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.150845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:43.152068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:43.179610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:43.221219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1266/2556] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:43.230360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.230853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.231446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.255654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:43.257995 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:43.266002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1291/2557] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:43.270286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.280681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775186-979a-4a47-a5bb-30e6360937fa None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.285321 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.285919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.286217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.286493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.287143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:43.288974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.289770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.301184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1267/2558] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:48:43.305513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.314780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa925f8-55eb-47ac-be1e-aaf77026304f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.319765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.320360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.320643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.320917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.321534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.323467 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.351424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1292/2559] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:43.355563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.365693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1502eb-4918-488b-8e7d-f11af55135cf admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.421689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.422336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.422620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.422890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.423518 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.425484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.447294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1268/2560] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:43 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:43.451570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.462637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5524f4b-00ee-4145-a28a-e6995e1c1434 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.467325 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.467924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.468203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.468474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.469136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.470859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.481442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1293/2561] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:43.485717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.495821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f620af50-cdcf-476c-b4ac-e10ce675c559 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.500534 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29260a-5aa3-49a7-a873-f19cd0057a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.501156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29260a-5aa3-49a7-a873-f19cd0057a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.501434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29260a-5aa3-49a7-a873-f19cd0057a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.501705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29260a-5aa3-49a7-a873-f19cd0057a25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.502376 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29260a-5aa3-49a7-a873-f19cd0057a25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.503981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29260a-5aa3-49a7-a873-f19cd0057a25 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.514977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1269/2562] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:43.519439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.530093 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b992b4a-bf5b-4527-a7bf-edd185eae96e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.534831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.535484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.535765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.536038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] PATH_INFO: `/v3/projects/667eb7d4acbf40b0ad03dd1eb41396c1/users/98cf9414b27c48c49b2f5d485d4689d5/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.551610 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] RBAC: Authorizing `identity:create_grant(project_id=667eb7d4acbf40b0ad03dd1eb41396c1, user_id=98cf9414b27c48c49b2f5d485d4689d5, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.554340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.568243 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1294/2563] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:43 2022] PUT /identity/v3/projects/667eb7d4acbf40b0ad03dd1eb41396c1/users/98cf9414b27c48c49b2f5d485d4689d5/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:43.583020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.583478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.583586 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.613845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] MFA Rules not processed for user `98cf9414b27c48c49b2f5d485d4689d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:43.615118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:43.678807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1270/2564] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:43.689970 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.691722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:43.750328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95f41ef-73c8-45c7-87e7-685ac4c3864a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.754087 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248c05b-ea33-4ecb-8923-636f5cf0b401 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.754720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c05b-ea33-4ecb-8923-636f5cf0b401 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.755002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c05b-ea33-4ecb-8923-636f5cf0b401 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.755277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248c05b-ea33-4ecb-8923-636f5cf0b401 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.764196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c05b-ea33-4ecb-8923-636f5cf0b401 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.765626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248c05b-ea33-4ecb-8923-636f5cf0b401 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.775073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1295/2565] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:43.793553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:43.824967 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:43.858679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804db388-50cf-41b5-ade5-9516ff1669cf None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:43.863276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:43.863903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.864184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.864457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] PATH_INFO: `/v3/users/98cf9414b27c48c49b2f5d485d4689d5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.866136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] RBAC: Authorizing `identity:delete_user(user_id=98cf9414b27c48c49b2f5d485d4689d5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:43.868164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:43.913008 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98cf9414b27c48c49b2f5d485d4689d5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.913679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98cf9414b27c48c49b2f5d485d4689d5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.914335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98cf9414b27c48c49b2f5d485d4689d5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.921197 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98cf9414b27c48c49b2f5d485d4689d5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.927820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98cf9414b27c48c49b2f5d485d4689d5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:43.946855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1271/2566] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:43 2022] DELETE /identity/v3/users/98cf9414b27c48c49b2f5d485d4689d5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:43.951343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:43.951629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:43.951902 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:43.970540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:43.971785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:44.030235 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1296/2567] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:44.040063 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.041485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:44.105873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80839295-bf36-4d3b-b60b-30524bff1acb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.110078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0f324-8120-4d09-97dc-c22e4a6cba7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.110420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f324-8120-4d09-97dc-c22e4a6cba7b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.110726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f324-8120-4d09-97dc-c22e4a6cba7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.110996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0f324-8120-4d09-97dc-c22e4a6cba7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.120835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0f324-8120-4d09-97dc-c22e4a6cba7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.122300 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0f324-8120-4d09-97dc-c22e4a6cba7b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.132529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1272/2568] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:44.396081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.398241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:44.410104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9926a927-0800-4381-b8b4-bc460a99f4a5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.414690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.415319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.415592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.415860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] PATH_INFO: `/v3/projects/667eb7d4acbf40b0ad03dd1eb41396c1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.416963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=667eb7d4acbf40b0ad03dd1eb41396c1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.418897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.430870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:44.471048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '667eb7d4acbf40b0ad03dd1eb41396c1', 'request_id': 'req-97496b60-fc28-440c-bd77-97fcc1cea818'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:44.476796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '667eb7d4acbf40b0ad03dd1eb41396c1', 'request_id': 'req-97496b60-fc28-440c-bd77-97fcc1cea818'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:44.482812 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '667eb7d4acbf40b0ad03dd1eb41396c1', 'request_id': 'req-97496b60-fc28-440c-bd77-97fcc1cea818'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:44.494582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] The token cache is being invalidate because project 667eb7d4acbf40b0ad03dd1eb41396c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:44.494862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:44.495481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-97496b60-fc28-440c-bd77-97fcc1cea818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:44.496935 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1297/2569] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:44 2022] DELETE /identity/v3/projects/667eb7d4acbf40b0ad03dd1eb41396c1 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:44.502000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.502305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.502574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.523539 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:44.524040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:44.566645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:44.586982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1273/2570] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:44.594375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.594730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.595027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.617190 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:44.618424 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:44.626170 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1298/2571] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:44.630369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.640722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c116b7-e3cc-4439-aa1e-00e8f5cd989b None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.645277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.645868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.646147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.646445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.647089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:44.647390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.648979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.660571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1274/2572] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:48:44.664897 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.674012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361c93f-8dc3-4afd-a0e7-b2cc0a809ac5 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.679251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.679841 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.680116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.680380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.680992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.682898 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.711057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1299/2573] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:44 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:44.715343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.725717 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1531c219-3ea3-44f6-9150-fceee82716be admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.788250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.788761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.789038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.789307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.789934 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.791884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.813303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1275/2574] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:44 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:44.817575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.828910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc81ce2-df8a-4cde-9d53-9dbca8eac49a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.833596 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.834215 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.834508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.834781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.835425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.837014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.847077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1300/2575] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:44.851161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.861033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26646ba6-5156-46f3-bd1f-34920eb7a626 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.866107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ba5517-c30d-425a-b462-2fd9ba351291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.866727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba5517-c30d-425a-b462-2fd9ba351291 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.867007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba5517-c30d-425a-b462-2fd9ba351291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.867276 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ba5517-c30d-425a-b462-2fd9ba351291 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.867917 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba5517-c30d-425a-b462-2fd9ba351291 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.869523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ba5517-c30d-425a-b462-2fd9ba351291 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.880749 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1276/2576] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:44.887152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:44.897694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2d75b9-178d-4481-bca7-158f594d860b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:44.903965 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:44.904698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.905034 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.905349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] PATH_INFO: `/v3/projects/1bcc086bb1ab4a68b386b8e8a1c41b9e/users/c32d4aa9f32048a5a4f37636aa7aee82/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.920464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bcc086bb1ab4a68b386b8e8a1c41b9e, user_id=c32d4aa9f32048a5a4f37636aa7aee82, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:44.923131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:44.937109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1301/2577] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:44 2022] PUT /identity/v3/projects/1bcc086bb1ab4a68b386b8e8a1c41b9e/users/c32d4aa9f32048a5a4f37636aa7aee82/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:44.948591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:44.948875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:44.949147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:44.980974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] MFA Rules not processed for user `c32d4aa9f32048a5a4f37636aa7aee82`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:44.982227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:45.063071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1277/2578] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:45.074846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:45.076807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:45.136847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc27f62-33b5-4ad2-888f-4122dc9d4259 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:45.140856 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21341b7f-a33f-4ec4-8d79-01f8e59ff661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:45.141462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341b7f-a33f-4ec4-8d79-01f8e59ff661 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:45.141740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341b7f-a33f-4ec4-8d79-01f8e59ff661 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:45.142007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21341b7f-a33f-4ec4-8d79-01f8e59ff661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:45.151359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21341b7f-a33f-4ec4-8d79-01f8e59ff661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:45.152875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21341b7f-a33f-4ec4-8d79-01f8e59ff661 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:45.162569 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1302/2579] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:45.184325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:45.232739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:45.250175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d3c335-64f1-4c18-a7c9-865ba9ab6b10 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:45.255070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:45.255701 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:45.256570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:45.256884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] PATH_INFO: `/v3/users/c32d4aa9f32048a5a4f37636aa7aee82` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:45.258163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c32d4aa9f32048a5a4f37636aa7aee82)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:45.260135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:45.314711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c32d4aa9f32048a5a4f37636aa7aee82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.315295 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c32d4aa9f32048a5a4f37636aa7aee82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.315915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c32d4aa9f32048a5a4f37636aa7aee82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.323690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c32d4aa9f32048a5a4f37636aa7aee82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.332997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c32d4aa9f32048a5a4f37636aa7aee82'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.355372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1278/2580] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:45 2022] DELETE /identity/v3/users/c32d4aa9f32048a5a4f37636aa7aee82 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:45.359972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:45.360251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:45.360528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:45.380324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:45.382285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:45.416037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:45.447473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1303/2581] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:45.458449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:45.459426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:45.522098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fe10c-991e-458c-b085-7121b06c3d33 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:45.526169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6295ec1c-bdf3-472e-a407-63a4c3450195 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:45.526822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295ec1c-bdf3-472e-a407-63a4c3450195 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:45.528117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295ec1c-bdf3-472e-a407-63a4c3450195 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:45.528117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295ec1c-bdf3-472e-a407-63a4c3450195 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:45.537564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295ec1c-bdf3-472e-a407-63a4c3450195 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:45.539046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6295ec1c-bdf3-472e-a407-63a4c3450195 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:45.548767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1279/2582] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:45.824220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:45.826388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:45.838350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2c6333-6814-4bcf-ae5f-889cfdbf1265 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:45.842969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:45.843595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:45.843868 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:45.844133 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] PATH_INFO: `/v3/projects/1bcc086bb1ab4a68b386b8e8a1c41b9e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:45.845704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bcc086bb1ab4a68b386b8e8a1c41b9e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:45.847657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:45.894140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bcc086bb1ab4a68b386b8e8a1c41b9e', 'request_id': 'req-cb2b9167-5d48-4156-a4be-b0fd903c14bf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.899742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bcc086bb1ab4a68b386b8e8a1c41b9e', 'request_id': 'req-cb2b9167-5d48-4156-a4be-b0fd903c14bf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.905952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bcc086bb1ab4a68b386b8e8a1c41b9e', 'request_id': 'req-cb2b9167-5d48-4156-a4be-b0fd903c14bf'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.919017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] The token cache is being invalidate because project 1bcc086bb1ab4a68b386b8e8a1c41b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:45.919302 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.919916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cb2b9167-5d48-4156-a4be-b0fd903c14bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:45.921399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1304/2583] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:45 2022] DELETE /identity/v3/projects/1bcc086bb1ab4a68b386b8e8a1c41b9e => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:45.926327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:45.926841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:45.926940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:45.947287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:45.948464 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:46.007344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1280/2584] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:46.016379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.016668 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.016945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.036991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:46.038225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:46.046060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1305/2585] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:46.050448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.059985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b6ba3f-b60b-4e9f-92c9-fd86126c9fce None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.064582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.065176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.065456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.065782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.066408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:46.066710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.068312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.070538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:46.085852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1281/2586] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:48:46.086899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.101015 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74464db-9c6b-48af-b6e7-5c9821419581 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.109060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.109346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.109655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.109931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.110612 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.112513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.141537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1306/2587] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:46 2022] POST /identity/v3/projects => generated 361 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:46.145661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.155932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958892c8-da12-419b-9a2c-55d677d7b864 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.212622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.213223 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.213505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.213777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.214452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.216391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.236718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1282/2588] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:46 2022] POST /identity/v3/users => generated 281 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:46.241480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.251863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ea5a8-9256-4829-90dc-448eecf9f861 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.257158 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.257754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.258032 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.258334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.259004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.260582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.270346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1307/2589] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:46.274812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.285096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f52ccdd-3228-4be3-8a9c-80c4682eb06a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.290353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e940994-8c68-411d-9dc2-e16636555c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.290977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e940994-8c68-411d-9dc2-e16636555c9b admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.291262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e940994-8c68-411d-9dc2-e16636555c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.291537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e940994-8c68-411d-9dc2-e16636555c9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.292183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e940994-8c68-411d-9dc2-e16636555c9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.293788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e940994-8c68-411d-9dc2-e16636555c9b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.304285 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1283/2590] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:46.308575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.318622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2cd7f-cece-4fa9-b283-1502a8c8607a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.323742 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.324393 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.324681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.324978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] PATH_INFO: `/v3/projects/9bb39d24505742dfa9f6ca055eb51040/users/75202f049acb4e71b82299a1eff2a398/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.340154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bb39d24505742dfa9f6ca055eb51040, user_id=75202f049acb4e71b82299a1eff2a398, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.342830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.356509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1308/2591] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:46 2022] PUT /identity/v3/projects/9bb39d24505742dfa9f6ca055eb51040/users/75202f049acb4e71b82299a1eff2a398/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:48:46.366692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.366978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.367253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.394178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] MFA Rules not processed for user `75202f049acb4e71b82299a1eff2a398`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:46.395435 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:46.458760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1284/2592] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:46.492525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.494238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:46.550878 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c78de36-d6fa-49d7-8d63-4d23d7eed740 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.554640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ff1586-aa18-4e6c-8a55-67b23ab63d3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.555246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff1586-aa18-4e6c-8a55-67b23ab63d3e service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.555528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff1586-aa18-4e6c-8a55-67b23ab63d3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.555803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ff1586-aa18-4e6c-8a55-67b23ab63d3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.564352 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff1586-aa18-4e6c-8a55-67b23ab63d3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.565817 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ff1586-aa18-4e6c-8a55-67b23ab63d3e service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.571301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:46.575958 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1309/2593] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:46.591894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.654301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70675c01-d462-4e2d-a9c3-de36b119184a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.658920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.659545 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.659829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.660100 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] PATH_INFO: `/v3/users/75202f049acb4e71b82299a1eff2a398` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.661238 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75202f049acb4e71b82299a1eff2a398)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.663211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.665501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:46.714868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75202f049acb4e71b82299a1eff2a398'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:46.715576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75202f049acb4e71b82299a1eff2a398'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:46.716198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75202f049acb4e71b82299a1eff2a398'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:46.723298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75202f049acb4e71b82299a1eff2a398'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:46.731405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75202f049acb4e71b82299a1eff2a398'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:46.748546 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1285/2594] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:46 2022] DELETE /identity/v3/users/75202f049acb4e71b82299a1eff2a398 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:46.752848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.753132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.753428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.774400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:46.775702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:46.836824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1310/2595] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:46.846476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:46.847880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:46.906649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef435c83-6523-4ad4-be3a-96445d043862 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:46.911277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80df8c6-1c62-4f87-b99d-9c24f1231dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:46.912044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80df8c6-1c62-4f87-b99d-9c24f1231dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:46.912477 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80df8c6-1c62-4f87-b99d-9c24f1231dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:46.912761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80df8c6-1c62-4f87-b99d-9c24f1231dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:46.923040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80df8c6-1c62-4f87-b99d-9c24f1231dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:46.924785 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80df8c6-1c62-4f87-b99d-9c24f1231dd4 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:46.935443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1286/2596] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:47.209047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.211500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:47.224343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93f2411-1ea2-405d-9a77-cb4a65203d71 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.229741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.230501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.230861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.231219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] PATH_INFO: `/v3/projects/9bb39d24505742dfa9f6ca055eb51040` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.232411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bb39d24505742dfa9f6ca055eb51040)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.234487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.282804 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bb39d24505742dfa9f6ca055eb51040', 'request_id': 'req-59300f77-4d18-46ae-b396-7e59343f9606'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:47.288250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bb39d24505742dfa9f6ca055eb51040', 'request_id': 'req-59300f77-4d18-46ae-b396-7e59343f9606'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:47.294444 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bb39d24505742dfa9f6ca055eb51040', 'request_id': 'req-59300f77-4d18-46ae-b396-7e59343f9606'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:47.306783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] The token cache is being invalidate because project 9bb39d24505742dfa9f6ca055eb51040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:47.307072 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:47.307706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59300f77-4d18-46ae-b396-7e59343f9606 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:47.309191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1311/2597] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:47 2022] DELETE /identity/v3/projects/9bb39d24505742dfa9f6ca055eb51040 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:47.314304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.314589 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.314864 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.335329 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:47.336538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:47.395835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1287/2598] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:47.404309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.404598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.404871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.424156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:47.425364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:47.433329 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1312/2599] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:47.437514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.447983 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2343752-ba07-4fb8-b52e-458e31fafbd3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.452628 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.453219 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.453496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.453767 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.454446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:47.454745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.456340 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.468861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1288/2600] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:48:47.473107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.482341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dcd2f91-1fdf-42a4-ab11-bb5aadeeaa8b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.486949 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.487545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.487825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.488098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.488729 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.490652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.518054 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1313/2601] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:47 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:47.522868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.532810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27104c-97c4-41c0-8ead-ce1341790cc6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.540452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:47.591928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.592537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.592817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.593089 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.593725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.595715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.618598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1289/2602] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:47 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:48:47.622752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.634067 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba2b74-9119-44bd-b100-de2f7cb0720c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.638695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.639304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.639585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.639859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.640524 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.642106 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.652024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1314/2603] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:47.655862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.665899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c5e4d-75e8-4f3b-bc18-2dbe8cdd1970 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.670541 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befae5d3-1ae8-40a4-b29b-21a0bd2e2b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.671136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befae5d3-1ae8-40a4-b29b-21a0bd2e2b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.671415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befae5d3-1ae8-40a4-b29b-21a0bd2e2b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.671688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befae5d3-1ae8-40a4-b29b-21a0bd2e2b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.672328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befae5d3-1ae8-40a4-b29b-21a0bd2e2b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.674009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befae5d3-1ae8-40a4-b29b-21a0bd2e2b42 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.684984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1290/2604] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:47.689012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.699947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc409b16-5ff8-4f63-abc3-a043cef6b9b8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.704658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.705334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.705624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.705894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] PATH_INFO: `/v3/projects/d71b5f9e3480459e90ceba653e4e00a3/users/382712d87f1947b6a5defe7db8741eb3/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.720604 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71b5f9e3480459e90ceba653e4e00a3, user_id=382712d87f1947b6a5defe7db8741eb3, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.723326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.737708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1315/2605] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:47 2022] PUT /identity/v3/projects/d71b5f9e3480459e90ceba653e4e00a3/users/382712d87f1947b6a5defe7db8741eb3/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:47.749576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.749931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.750298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.782802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] MFA Rules not processed for user `382712d87f1947b6a5defe7db8741eb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:47.784038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:47.856378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1291/2606] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:47.868146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:47.869948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:47.892525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:47.929261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6177dc-3652-4ee9-ad67-6de285726c67 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:47.932980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdac021-b090-4bbb-bb4f-72c1a0277cf5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:47.933581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac021-b090-4bbb-bb4f-72c1a0277cf5 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:47.933871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac021-b090-4bbb-bb4f-72c1a0277cf5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:47.934140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdac021-b090-4bbb-bb4f-72c1a0277cf5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:47.942528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdac021-b090-4bbb-bb4f-72c1a0277cf5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:47.944943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdac021-b090-4bbb-bb4f-72c1a0277cf5 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:47.953518 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1316/2607] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:47.978216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:48.054457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9107e6-1eda-450a-bae4-bec0c808a97d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:48.059380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:48.060026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.060309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.060647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] PATH_INFO: `/v3/users/382712d87f1947b6a5defe7db8741eb3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.061840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=382712d87f1947b6a5defe7db8741eb3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:48.063865 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:48.095218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:48.133390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382712d87f1947b6a5defe7db8741eb3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.134212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382712d87f1947b6a5defe7db8741eb3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.134916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382712d87f1947b6a5defe7db8741eb3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.142583 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382712d87f1947b6a5defe7db8741eb3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.150114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382712d87f1947b6a5defe7db8741eb3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.167825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1292/2608] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:47 2022] DELETE /identity/v3/users/382712d87f1947b6a5defe7db8741eb3 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:48.172412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.172695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.172974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.192102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:48.193233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:48.253775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1317/2609] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:48.264383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:48.265850 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:48.326885 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2e49ac-4c87-4005-9565-52f0e1730b9a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:48.330761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236436aa-1437-4c59-97fb-1c7aca2f4272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:48.331928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236436aa-1437-4c59-97fb-1c7aca2f4272 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.331928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236436aa-1437-4c59-97fb-1c7aca2f4272 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.331928 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236436aa-1437-4c59-97fb-1c7aca2f4272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.342434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236436aa-1437-4c59-97fb-1c7aca2f4272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:48.343877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236436aa-1437-4c59-97fb-1c7aca2f4272 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:48.353923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1293/2610] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:48.630955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:48.633196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:48.645601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b9c733-f007-49fb-8608-386a32f849c6 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:48.650924 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:48.651561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.651844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.652117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] PATH_INFO: `/v3/projects/d71b5f9e3480459e90ceba653e4e00a3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.653270 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71b5f9e3480459e90ceba653e4e00a3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:48.655230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:48.702399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71b5f9e3480459e90ceba653e4e00a3', 'request_id': 'req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.707647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71b5f9e3480459e90ceba653e4e00a3', 'request_id': 'req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.714025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71b5f9e3480459e90ceba653e4e00a3', 'request_id': 'req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.727514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] The token cache is being invalidate because project d71b5f9e3480459e90ceba653e4e00a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:48.727800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.728412 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-59d4d431-a84f-48e3-b41a-a0c87f4f16e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:48.729880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1318/2611] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:48 2022] DELETE /identity/v3/projects/d71b5f9e3480459e90ceba653e4e00a3 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:48.734715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.734999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.735273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.755131 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:48.756503 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:48.820336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1294/2612] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:48.829637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.829979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.830285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.854951 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:48.856169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:48.864228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1319/2613] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:48.869457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:48.879415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a4ddca-6646-47b6-bb04-3b05894f96d2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:48.884540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:48.885173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.885455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.885726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.886414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:48.886719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:48.888686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:48.905842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1295/2614] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:48:48.906832 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:48.920254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149991e6-fa7a-421f-a038-1d5ef620c56c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:48.930531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:48.930531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:48.930531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:48.930531 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:48.931222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:48.931222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:48.967389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1320/2615] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:48 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:48.968923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:48.982565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb355678-221c-420a-a9d0-419dabca9a1d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:49.017221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:49.055957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:49.056580 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.056858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.057128 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:49.057768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:49.059769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:49.082560 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1296/2616] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:48 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:49.087357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:49.098102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30957dea-7cc9-445b-8d0d-639fcd7c458b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:49.103172 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:49.103783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.104063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.104337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:49.105014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:49.106749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:49.117790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1321/2617] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:49.122016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:49.132639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dce61a1-37b3-4516-95eb-f6cc9bd85c31 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:49.137236 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5de9a-1031-402e-8fd6-ea4faa6ec40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:49.137825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5de9a-1031-402e-8fd6-ea4faa6ec40e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.138101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5de9a-1031-402e-8fd6-ea4faa6ec40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.138402 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5de9a-1031-402e-8fd6-ea4faa6ec40e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:49.139035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5de9a-1031-402e-8fd6-ea4faa6ec40e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:49.140627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5de9a-1031-402e-8fd6-ea4faa6ec40e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:49.151052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1297/2618] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:49.155049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:49.165088 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4b5e25-571e-4860-85e6-7f230fde3521 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:49.171002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:49.171654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.171942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.172216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] PATH_INFO: `/v3/projects/9b17dad9df1840cc816875bcc42f1706/users/bec33f84fef94b56ab0e1639bb8f9b14/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:49.174643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:49.190558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b17dad9df1840cc816875bcc42f1706, user_id=bec33f84fef94b56ab0e1639bb8f9b14, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:49.193230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:49.207737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1322/2619] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:49 2022] PUT /identity/v3/projects/9b17dad9df1840cc816875bcc42f1706/users/bec33f84fef94b56ab0e1639bb8f9b14/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:49.213212 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6698ed-678a-4eab-88a6-c86b99ff5868 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.213553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6698ed-678a-4eab-88a6-c86b99ff5868 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.213829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6698ed-678a-4eab-88a6-c86b99ff5868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:49.234105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5d6698ed-678a-4eab-88a6-c86b99ff5868 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:49.235422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5d6698ed-678a-4eab-88a6-c86b99ff5868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:49.296768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1298/2620] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:49.307162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:49.308867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:49.369908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97def61f-dc96-4a51-9fd7-f87bf26e52bf admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:49.373508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:49.374111 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.374414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.374790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:49.384695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:49.386109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:49.395279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1323/2621] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:49.991893 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:49.992391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:49.992516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:50.022328 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] MFA Rules not processed for user `bec33f84fef94b56ab0e1639bb8f9b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:50.023554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:50.097825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1299/2622] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:49 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:50.109248 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:50.111794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:50.193389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92754b0-a584-4d1e-ac23-b76b78c6bb14 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:50.197535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07785f0-8ac0-473d-9edd-65209fe8f6d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:50.198304 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07785f0-8ac0-473d-9edd-65209fe8f6d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:50.198736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07785f0-8ac0-473d-9edd-65209fe8f6d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:50.199345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07785f0-8ac0-473d-9edd-65209fe8f6d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:50.208083 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07785f0-8ac0-473d-9edd-65209fe8f6d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:50.210255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07785f0-8ac0-473d-9edd-65209fe8f6d9 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:50.221486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1324/2623] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:50 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:50.224496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:50.224496 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:50.237664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9945c774-eb89-42dd-a916-7cda9780650f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:50.242626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:50.243263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:50.243543 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:50.243817 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] PATH_INFO: `/v3/users/85d486bc02cb460e8007ec67b5cdb0f2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:50.244970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85d486bc02cb460e8007ec67b5cdb0f2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:50.246944 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:50.286509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:50.300695 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85d486bc02cb460e8007ec67b5cdb0f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:50.301367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85d486bc02cb460e8007ec67b5cdb0f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:50.302033 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85d486bc02cb460e8007ec67b5cdb0f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:50.310415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85d486bc02cb460e8007ec67b5cdb0f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:50.319433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85d486bc02cb460e8007ec67b5cdb0f2'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:50.338026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1300/2624] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:48:50 2022] DELETE /identity/v3/users/85d486bc02cb460e8007ec67b5cdb0f2 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:50.342704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79762683-2870-4e12-956a-40d83d44487f None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:50.342991 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79762683-2870-4e12-956a-40d83d44487f None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:50.343263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79762683-2870-4e12-956a-40d83d44487f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:50.362511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-79762683-2870-4e12-956a-40d83d44487f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:50.363733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-79762683-2870-4e12-956a-40d83d44487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:50.424226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1325/2625] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:50.435500 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:50.436978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:50.501022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422be2e-d0a9-4ebb-8220-98bd03294f44 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:50.504995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9f7a79-1e2d-4560-8653-446ea488369a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:50.505612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7a79-1e2d-4560-8653-446ea488369a service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:50.505890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7a79-1e2d-4560-8653-446ea488369a service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:50.506935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9f7a79-1e2d-4560-8653-446ea488369a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:50.518348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7a79-1e2d-4560-8653-446ea488369a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:50.520151 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9f7a79-1e2d-4560-8653-446ea488369a service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:50.532530 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1301/2626] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:48:51.072494 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79762683-2870-4e12-956a-40d83d44487f None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.073948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-79762683-2870-4e12-956a-40d83d44487f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:51.088408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79762683-2870-4e12-956a-40d83d44487f None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.093118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.093764 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.094041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.094353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] PATH_INFO: `/v3/projects/38912b0899834adb8988b455bdfcfd27` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.096239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] RBAC: Authorizing `identity:delete_project(project_id=38912b0899834adb8988b455bdfcfd27)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.098168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.124456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:51.150673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38912b0899834adb8988b455bdfcfd27', 'request_id': 'req-57ac68a3-db1b-4256-baad-1635c62a997a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:51.156246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38912b0899834adb8988b455bdfcfd27', 'request_id': 'req-57ac68a3-db1b-4256-baad-1635c62a997a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:51.162431 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38912b0899834adb8988b455bdfcfd27', 'request_id': 'req-57ac68a3-db1b-4256-baad-1635c62a997a'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:51.175006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] The token cache is being invalidate because project 38912b0899834adb8988b455bdfcfd27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:48:51.175298 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:51.175927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57ac68a3-db1b-4256-baad-1635c62a997a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:48:51.177428 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1326/2627] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:48:51 2022] DELETE /identity/v3/projects/38912b0899834adb8988b455bdfcfd27 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:51.182422 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.182884 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.182996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.203011 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:51.204153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:51.262862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1302/2628] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:51.272362 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.272732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.273078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.292778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:51.294048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:51.302600 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1327/2629] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:51.306857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.317373 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02129d9f-d2d1-4944-8a97-aa8e6f139376 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.322849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.323450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.323727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.323994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.324637 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:48:51.324938 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.326570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.339065 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1303/2630] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:48:51.343484 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.352861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ed0fa8-af5a-4d58-9578-4ebfb0d17238 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.357537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.358138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.358453 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.358726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.359349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.361247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.389538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1328/2631] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:51.393699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.403866 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5028e7e-7f06-409b-9b0c-b4eda6b93835 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.442769 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:51.463922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.464538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.464819 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.465096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.465740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.467739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.490173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1304/2632] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:51.494831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.505042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea330a1-ffc3-459f-b324-cee9de9de9fd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.509605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.510225 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.510505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.510772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.511405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.513011 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.523480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1329/2633] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:51.527454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.537599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a6e908-8fa5-40e8-8df8-ddea341529b0 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.542423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f39f53-a30f-4688-b41f-883c107836e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.543060 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39f53-a30f-4688-b41f-883c107836e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.543336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39f53-a30f-4688-b41f-883c107836e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.543604 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f39f53-a30f-4688-b41f-883c107836e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.544245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f39f53-a30f-4688-b41f-883c107836e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.545833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f39f53-a30f-4688-b41f-883c107836e0 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.557279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1305/2634] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:51.561090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.572066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33d05ab-6849-4d4f-b667-b330b9b48ffe admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.582777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54542de-881e-40cd-82a7-1766e7f16b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.582777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54542de-881e-40cd-82a7-1766e7f16b99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.582777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54542de-881e-40cd-82a7-1766e7f16b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.582777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54542de-881e-40cd-82a7-1766e7f16b99 admin admin] PATH_INFO: `/v3/projects/49c77f318efb4ac8a1430e12b976c22c/users/2cf08d12b85946aaa4282c96465d87e9/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.593199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54542de-881e-40cd-82a7-1766e7f16b99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49c77f318efb4ac8a1430e12b976c22c, user_id=2cf08d12b85946aaa4282c96465d87e9, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.596216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54542de-881e-40cd-82a7-1766e7f16b99 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.596830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.597270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.597554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.609480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1330/2635] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:51 2022] PUT /identity/v3/projects/49c77f318efb4ac8a1430e12b976c22c/users/2cf08d12b85946aaa4282c96465d87e9/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:51.619961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] MFA Rules not processed for user `bec33f84fef94b56ab0e1639bb8f9b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:51.620867 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.621149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.621252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:51.621421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.650448 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] MFA Rules not processed for user `2cf08d12b85946aaa4282c96465d87e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:51.651644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:51.679269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1306/2636] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:48:51.723321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1331/2637] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:51.728359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.797111 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637d6960-6a9f-4cf9-8b0b-63dce665af23 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.801812 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.802454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.802736 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.803036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.803681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.805678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.838169 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1307/2638] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:51.839410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.851343 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0df8e-7907-41fb-b908-5287a4559ef0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.886838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:51.914504 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.915123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.915464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.915743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.916364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.918336 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.940183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1332/2639] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:48:51 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:48:51.944660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.955488 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b63eeb-d3e2-41fb-a03b-b17f90a0f253 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:51.960903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:51.961447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:51.961715 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:51.962107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:51.962798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:51.964415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:51.980709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1308/2640] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:51.985000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:51.995221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e21ab2e-43ff-4dd2-b965-2bb07fbd021d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:52.000618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:52.001300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.001587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.001861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] PATH_INFO: `/v3/projects/542ac6231c3e47d7b64708267c2c6e72/users/f6c9d38170eb45ab8e78e620bf14b9aa/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.022677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=542ac6231c3e47d7b64708267c2c6e72, user_id=f6c9d38170eb45ab8e78e620bf14b9aa, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:52.025659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:52.041682 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1333/2641] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:51 2022] PUT /identity/v3/projects/542ac6231c3e47d7b64708267c2c6e72/users/f6c9d38170eb45ab8e78e620bf14b9aa/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:52.045509 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:52.055714 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0785fa-a4a8-4bcf-bd60-bbf369f59890 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:52.091782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:48:52.125484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1d7697-f335-4bc6-9636-9d45f3ccdf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:52.126107 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d7697-f335-4bc6-9636-9d45f3ccdf33 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.126406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d7697-f335-4bc6-9636-9d45f3ccdf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.126822 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1d7697-f335-4bc6-9636-9d45f3ccdf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.127429 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d7697-f335-4bc6-9636-9d45f3ccdf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:52.129483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1d7697-f335-4bc6-9636-9d45f3ccdf33 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:52.143342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1309/2642] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:48:52.144458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:52.157626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a76c05-3351-4907-a958-3afa43c4fcbb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:52.166844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:52.168013 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.168528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.169021 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] PATH_INFO: `/v3/projects/542ac6231c3e47d7b64708267c2c6e72/users/f6c9d38170eb45ab8e78e620bf14b9aa/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.172068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=542ac6231c3e47d7b64708267c2c6e72, user_id=f6c9d38170eb45ab8e78e620bf14b9aa, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:52.174744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:52.185933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1334/2643] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:48:52 2022] PUT /identity/v3/projects/542ac6231c3e47d7b64708267c2c6e72/users/f6c9d38170eb45ab8e78e620bf14b9aa/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:48:52.197172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.197954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.197954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.237987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] MFA Rules not processed for user `f6c9d38170eb45ab8e78e620bf14b9aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:48:52.239254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:52.269595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:52.272220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:52.328523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1310/2644] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:48:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:48:52.341989 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:52.343458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:52.378123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3b112-1798-40dc-894a-cc189011d259 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:52.384645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:52.385278 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.385563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.385824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.387710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:48:52.400311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:52.401921 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:52.408856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a1368-7f5a-4b57-b6f3-730073d8f48e None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:52.413345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:52.414146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.414498 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1335/2645] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:48:52.414665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.414821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.425185 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:52.426680 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:52.437954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1311/2646] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:48:52.525090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:48:52.535474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1f0a43-01a5-4413-8658-b644f955c169 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:48:52.539772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ec82e-2a4a-4f57-80e7-15984b816188 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:48:52.540375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ec82e-2a4a-4f57-80e7-15984b816188 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:48:52.540647 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ec82e-2a4a-4f57-80e7-15984b816188 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:48:52.540912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ec82e-2a4a-4f57-80e7-15984b816188 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:48:52.609823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ec82e-2a4a-4f57-80e7-15984b816188 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:48:52.611314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ec82e-2a4a-4f57-80e7-15984b816188 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:48:52.621632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1336/2647] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:48:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:02.433036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=72 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:02.617075 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=87,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:03.668681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:03.734941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23109e-66de-4475-86cb-161339986a9a service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:03.739899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:03.740600 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:03.740882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:03.741227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] PATH_INFO: `/v3/users/2cf08d12b85946aaa4282c96465d87e9` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:03.742577 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf08d12b85946aaa4282c96465d87e9)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:03.744891 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:03.792397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf08d12b85946aaa4282c96465d87e9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:03.793059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf08d12b85946aaa4282c96465d87e9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:03.793670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf08d12b85946aaa4282c96465d87e9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:03.800773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf08d12b85946aaa4282c96465d87e9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:03.808624 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf08d12b85946aaa4282c96465d87e9'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:03.829004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1312/2648] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:03 2022] DELETE /identity/v3/users/2cf08d12b85946aaa4282c96465d87e9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:03.833671 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:03.834226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:03.834226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:03.854938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:03.856177 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:03.917827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1337/2649] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:03.927979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:03.930384 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:03.996344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e7309d-37b2-4917-a5f2-609f0b8c9eba admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:04.000098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:04.000711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:04.000993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:04.001269 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:04.010889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:04.012607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:04.023076 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1313/2650] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:04.408324 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:04.410864 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:04.432823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac9272-f76f-4030-b92e-bf2527705abc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:04.437826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:04.438342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:04.438619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:04.438920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] PATH_INFO: `/v3/projects/49c77f318efb4ac8a1430e12b976c22c` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:04.439998 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49c77f318efb4ac8a1430e12b976c22c)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:04.441916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:04.492783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49c77f318efb4ac8a1430e12b976c22c', 'request_id': 'req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.498777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49c77f318efb4ac8a1430e12b976c22c', 'request_id': 'req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.505258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49c77f318efb4ac8a1430e12b976c22c', 'request_id': 'req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.519292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] The token cache is being invalidate because project 49c77f318efb4ac8a1430e12b976c22c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:04.519534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.520125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.521664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1338/2651] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:04 2022] DELETE /identity/v3/projects/49c77f318efb4ac8a1430e12b976c22c => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:04.526648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:04.536833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35010a0f-c8bd-414b-acd9-c2654b24af3d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:04.556626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:04.606201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:04.606860 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:04.607142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:04.607413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] PATH_INFO: `/v3/users/f6c9d38170eb45ab8e78e620bf14b9aa` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:04.608584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6c9d38170eb45ab8e78e620bf14b9aa)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:04.610571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:04.667234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6c9d38170eb45ab8e78e620bf14b9aa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.668097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6c9d38170eb45ab8e78e620bf14b9aa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.668881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6c9d38170eb45ab8e78e620bf14b9aa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.679734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6c9d38170eb45ab8e78e620bf14b9aa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.689337 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6c9d38170eb45ab8e78e620bf14b9aa'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:04.716922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1314/2652] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:04 2022] DELETE /identity/v3/users/f6c9d38170eb45ab8e78e620bf14b9aa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:05.020259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.023983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:05.040245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1bdf31-7ea7-42de-8092-5e96ad9fb363 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.101913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.103378 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:05.123655 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.124532 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.124901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.125245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] PATH_INFO: `/v3/projects/542ac6231c3e47d7b64708267c2c6e72` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.126657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=542ac6231c3e47d7b64708267c2c6e72)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.129212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.172646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:05.197746 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '542ac6231c3e47d7b64708267c2c6e72', 'request_id': 'req-1176d931-4065-4210-ac9b-77051b9bda0c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:05.205517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee9a39-b84a-43b9-aeb8-b9fd3e005ada admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.210722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '542ac6231c3e47d7b64708267c2c6e72', 'request_id': 'req-1176d931-4065-4210-ac9b-77051b9bda0c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:05.228452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '542ac6231c3e47d7b64708267c2c6e72', 'request_id': 'req-1176d931-4065-4210-ac9b-77051b9bda0c'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:05.247143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] The token cache is being invalidate because project 542ac6231c3e47d7b64708267c2c6e72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:05.247443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:05.248443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-1176d931-4065-4210-ac9b-77051b9bda0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:05.250309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1339/2653] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:05 2022] DELETE /identity/v3/projects/542ac6231c3e47d7b64708267c2c6e72 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:05.254798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.255072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:05.255223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.266573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.287925 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:05.289646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:05.290940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.291669 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.292025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.292342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.294853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:05.308857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.310528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2751885-4098-47f0-87ca-dd726990aec0 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.325381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1315/2654] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:49:05.376751 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1340/2655] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:49:05.391841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.392124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.392401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.415221 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:05.416442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:05.424642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1316/2656] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:05.430378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.449069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0419d7-6d61-4b38-bd4c-377f4de27a1b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.455408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.456030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.456318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.456595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.457253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:49:05.457558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.459216 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.473704 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1341/2657] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:49:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:49:05.478424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.489147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42fe1c5-756b-41d9-afed-a560ecc7546f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.495572 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.496224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.496641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.496915 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.497552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.499501 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.534170 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1317/2658] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:49:05 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:05.539623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.555220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55043c93-13a0-4a3d-b0cf-939917616839 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.626063 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.626732 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.627041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.627306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.627943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.629952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.653390 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1342/2659] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:49:05 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:05.657569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.670225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afabc53-352b-4426-86d3-1e0cf6aced2e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.679839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.681195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.681195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.681195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.682945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.683725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.696042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1318/2660] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:05.700134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.710848 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bc20f-bf14-4977-9192-96b0edf62bd7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.715981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bbfd08-57ff-4b8f-a7a6-de24b0532a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.716595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfd08-57ff-4b8f-a7a6-de24b0532a82 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.716873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfd08-57ff-4b8f-a7a6-de24b0532a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.717144 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bbfd08-57ff-4b8f-a7a6-de24b0532a82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.717785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbfd08-57ff-4b8f-a7a6-de24b0532a82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.719425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bbfd08-57ff-4b8f-a7a6-de24b0532a82 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.732918 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1343/2661] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:05.737724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:05.749613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e204bd-a6f1-4ff9-bd94-399a92073f54 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:05.754970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:05.755662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.755942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.756270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] PATH_INFO: `/v3/projects/d96d1bff128349a19483b34cc0df4757/users/17f6a6e497934360ab835832d493e7ef/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.773434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d96d1bff128349a19483b34cc0df4757, user_id=17f6a6e497934360ab835832d493e7ef, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:05.776172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:05.793721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1319/2662] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:05 2022] PUT /identity/v3/projects/d96d1bff128349a19483b34cc0df4757/users/17f6a6e497934360ab835832d493e7ef/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:05.805000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:05.805284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:05.805546 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:05.961460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] MFA Rules not processed for user `17f6a6e497934360ab835832d493e7ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:05.962731 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:05.971435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:06.031708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1344/2663] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:06.036324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.104842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f1886c-a7c4-41f1-a460-58f23901c31a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.109523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.110144 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.110474 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.110712 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.111387 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.113317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.142957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1320/2664] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:49:06 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:49:06.147866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.159310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7501edc-ae26-451a-afbc-9c9c50ea5792 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.221373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.221994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.222312 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.222584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.223226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.225199 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.247294 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1345/2665] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:49:06 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:49:06.251657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.263697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9b31d-bab2-4567-a761-a315bb81b5e2 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.268702 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.269313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.269592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.269863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.270552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.272918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.284514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1321/2666] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:06.288942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.299033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8387fc-fd23-41fb-a8da-4a33cf710040 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.303681 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.304330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.304606 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.304877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] PATH_INFO: `/v3/projects/ebc559cc026744d5a66c298665e9c20a/users/7d556991c0d04f028d564353cd3e74cc/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.320723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc559cc026744d5a66c298665e9c20a, user_id=7d556991c0d04f028d564353cd3e74cc, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.323423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.339185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1346/2667] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:06 2022] PUT /identity/v3/projects/ebc559cc026744d5a66c298665e9c20a/users/7d556991c0d04f028d564353cd3e74cc/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:06.343327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.355997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43449b7-7251-44e3-a66d-81310ac0f76e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.397711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:06.433766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a51c48-91bb-4b4c-911a-66d7ba3b626c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.433766 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a51c48-91bb-4b4c-911a-66d7ba3b626c admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.434972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a51c48-91bb-4b4c-911a-66d7ba3b626c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.434972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a51c48-91bb-4b4c-911a-66d7ba3b626c admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.435597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a51c48-91bb-4b4c-911a-66d7ba3b626c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.437225 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a51c48-91bb-4b4c-911a-66d7ba3b626c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.449017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1322/2668] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:06 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:06.453186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.464313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdc140c-3064-466c-b28b-42bc4417f0ea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.468963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.469627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.470138 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.470609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] PATH_INFO: `/v3/projects/ebc559cc026744d5a66c298665e9c20a/users/7d556991c0d04f028d564353cd3e74cc/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.473320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebc559cc026744d5a66c298665e9c20a, user_id=7d556991c0d04f028d564353cd3e74cc, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.476479 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.489850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1347/2669] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:06 2022] PUT /identity/v3/projects/ebc559cc026744d5a66c298665e9c20a/users/7d556991c0d04f028d564353cd3e74cc/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:06.501936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.502246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.502528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.532742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] MFA Rules not processed for user `7d556991c0d04f028d564353cd3e74cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:06.534246 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:06.627602 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1323/2670] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:49:06.641274 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.643171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:06.704157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd4a076-985a-47ef-b972-95c1f7a4dc80 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.708066 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.708691 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.708975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.709247 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.719656 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.721102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.731374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1348/2671] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:06.817147 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:06.832073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11218f8-905d-422a-9a3f-6635785a94fb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:06.836201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd1e3e3-6bd7-496d-a4e3-6f8cdb0e8a6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:06.837827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1e3e3-6bd7-496d-a4e3-6f8cdb0e8a6b service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:06.837827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1e3e3-6bd7-496d-a4e3-6f8cdb0e8a6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:06.837827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd1e3e3-6bd7-496d-a4e3-6f8cdb0e8a6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:06.900382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1e3e3-6bd7-496d-a4e3-6f8cdb0e8a6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:06.901862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd1e3e3-6bd7-496d-a4e3-6f8cdb0e8a6b service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:06.912087 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1324/2672] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:16.727147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:16.907229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=75 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:20.719986 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:20.786685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781dcc23-29ae-4e92-9663-65f4d64375a8 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:20.791271 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:20.791909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:20.792210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:20.792465 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] PATH_INFO: `/v3/users/17f6a6e497934360ab835832d493e7ef` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:20.793629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17f6a6e497934360ab835832d493e7ef)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:20.795617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:20.849752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17f6a6e497934360ab835832d493e7ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:20.850860 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17f6a6e497934360ab835832d493e7ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:20.851639 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17f6a6e497934360ab835832d493e7ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:20.860566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17f6a6e497934360ab835832d493e7ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:20.877754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17f6a6e497934360ab835832d493e7ef'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:20.892549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1349/2673] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:20 2022] DELETE /identity/v3/users/17f6a6e497934360ab835832d493e7ef => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:49:20.898875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:20.899405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:20.900118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:20.930679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:20.930679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:20.996497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1325/2674] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:21.007768 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:21.009208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:21.068837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b913c-50e9-4069-b1d4-60c4cd548348 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:21.073446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:21.074076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:21.074382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:21.074657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:21.084419 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:21.086550 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:21.096674 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1350/2675] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:21.415194 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:21.416679 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:21.433880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfaa9bc-8605-40d8-846f-da47619d6f94 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:21.450304 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:21.451014 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:21.451264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:21.451460 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] PATH_INFO: `/v3/projects/d96d1bff128349a19483b34cc0df4757` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:21.452752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d96d1bff128349a19483b34cc0df4757)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:21.454662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:21.521380 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd96d1bff128349a19483b34cc0df4757', 'request_id': 'req-02172790-f69c-4b0e-892b-ecce4c17159b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.526841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd96d1bff128349a19483b34cc0df4757', 'request_id': 'req-02172790-f69c-4b0e-892b-ecce4c17159b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.535272 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd96d1bff128349a19483b34cc0df4757', 'request_id': 'req-02172790-f69c-4b0e-892b-ecce4c17159b'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.550444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] The token cache is being invalidate because project d96d1bff128349a19483b34cc0df4757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:21.550982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.552000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.554759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1326/2676] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:21 2022] DELETE /identity/v3/projects/d96d1bff128349a19483b34cc0df4757 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:21.558778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:21.569667 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548258cd-5524-44f4-96a3-120afbcb794d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:21.589977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:21.635775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:21.636445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:21.636719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:21.636992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] PATH_INFO: `/v3/users/7d556991c0d04f028d564353cd3e74cc` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:21.638175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d556991c0d04f028d564353cd3e74cc)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:21.640135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:21.692049 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d556991c0d04f028d564353cd3e74cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.693033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d556991c0d04f028d564353cd3e74cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.693923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d556991c0d04f028d564353cd3e74cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.703450 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d556991c0d04f028d564353cd3e74cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.719587 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-94262a29-a886-4a9c-9d9c-1ca821b7902f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d556991c0d04f028d564353cd3e74cc'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:21.736223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1351/2677] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:21 2022] DELETE /identity/v3/users/7d556991c0d04f028d564353cd3e74cc => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:22.040213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.042289 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:22.064809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02172790-f69c-4b0e-892b-ecce4c17159b admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.258313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.258972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.259252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.260028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] PATH_INFO: `/v3/projects/ebc559cc026744d5a66c298665e9c20a` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.260727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebc559cc026744d5a66c298665e9c20a)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.262730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.295490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:22.315900 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebc559cc026744d5a66c298665e9c20a', 'request_id': 'req-373465ab-7513-4b0d-aad1-fd7b03186f07'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:22.323025 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebc559cc026744d5a66c298665e9c20a', 'request_id': 'req-373465ab-7513-4b0d-aad1-fd7b03186f07'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:22.331026 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebc559cc026744d5a66c298665e9c20a', 'request_id': 'req-373465ab-7513-4b0d-aad1-fd7b03186f07'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:22.343119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] The token cache is being invalidate because project ebc559cc026744d5a66c298665e9c20a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:22.343408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:22.344084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-373465ab-7513-4b0d-aad1-fd7b03186f07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:22.345625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1327/2678] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:22 2022] DELETE /identity/v3/projects/ebc559cc026744d5a66c298665e9c20a => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:22.350605 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.350936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.351222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.372399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:22.373554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:22.439523 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1352/2679] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:22.448902 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.449319 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.449597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.472266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:22.473502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:22.482828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1328/2680] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:22.487251 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.498560 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a033233-82af-40a4-ba34-584a2cd01e50 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.503808 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.504503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.504775 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.505039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.505689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:49:22.505981 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.507927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.521229 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1353/2681] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:49:22.525918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.536270 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fe6e25-d70c-4d2f-a800-2181258618dc None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.541873 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.542536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.542792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.543057 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.543691 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.545610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.577385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1329/2682] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:49:22 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:22.583445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.595499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd369f2-b441-4e19-a64a-2887b6d9ecb9 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.625196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:22.662117 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.662779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.663060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.663985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.664502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.667445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.692071 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1354/2683] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:49:22 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:49:22.696598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.713177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9a922a-c166-4a44-aaf4-c8ff3db1a32a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.718693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.719299 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.719571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.719877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.720622 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.722457 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.734259 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1330/2684] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:49:22.738358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.749378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e7308b-fce7-4b92-8d61-7b0a958da4e8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.754628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759556f4-496a-4eb8-8ccf-e6034914aa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.755242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759556f4-496a-4eb8-8ccf-e6034914aa9e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.755519 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759556f4-496a-4eb8-8ccf-e6034914aa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.755786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759556f4-496a-4eb8-8ccf-e6034914aa9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.756422 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759556f4-496a-4eb8-8ccf-e6034914aa9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.758102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759556f4-496a-4eb8-8ccf-e6034914aa9e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.772081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1355/2685] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:22.776406 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:22.787274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0387e38-b18e-4d3d-846c-ee44d83fb8d1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:22.792996 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:22.793665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.793946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.794245 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] PATH_INFO: `/v3/projects/f447662b94ea464790994cb233d8a16f/users/7c5129b1ac54492984a5d0321b0fedec/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.810423 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f447662b94ea464790994cb233d8a16f, user_id=7c5129b1ac54492984a5d0321b0fedec, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:22.813109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:22.828096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1331/2686] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:22 2022] PUT /identity/v3/projects/f447662b94ea464790994cb233d8a16f/users/7c5129b1ac54492984a5d0321b0fedec/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:22.840311 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:22.840592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:22.840861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:22.872574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] MFA Rules not processed for user `7c5129b1ac54492984a5d0321b0fedec`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:22.873795 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:22.952349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1356/2687] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:22.958218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.056149 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e2b493-a4dc-4a40-9b65-c6e84dae1794 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.060908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.061582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.061921 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.062216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.062840 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.064759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.093618 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1332/2688] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:49:22 2022] POST /identity/v3/projects => generated 363 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:49:23.098513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.116185 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efd176f-eaa4-4b79-88c6-5dd20c1ba775 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.180942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.181466 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.181741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.182007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.182666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.184627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.209855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1357/2689] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:49:23 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:23.214975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.231562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9feecf-a085-4e8e-9833-ca322a8b9828 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.236326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.236935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.238254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.238254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.238254 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.239807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.251673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1333/2690] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:23.256418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.267598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3675ef4e-e2b2-4d55-a7a0-db79151d79e1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.272380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.273640 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.273916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.274209 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] PATH_INFO: `/v3/projects/97ac54fbdc6147db8c6cfd5ab9d8cc19/users/e5cd83b9daa649e39219a394bc15d3fd/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.290786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ac54fbdc6147db8c6cfd5ab9d8cc19, user_id=e5cd83b9daa649e39219a394bc15d3fd, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.293793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.311880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1358/2691] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:23 2022] PUT /identity/v3/projects/97ac54fbdc6147db8c6cfd5ab9d8cc19/users/e5cd83b9daa649e39219a394bc15d3fd/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:49:23.317043 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.329581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55f5319-c69e-4c51-ab1a-f4362b3ab3ea admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.352472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:23.400349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4643682b-7992-4297-92c0-a75efd97332f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.400994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643682b-7992-4297-92c0-a75efd97332f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.401377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643682b-7992-4297-92c0-a75efd97332f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.401599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4643682b-7992-4297-92c0-a75efd97332f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.402229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643682b-7992-4297-92c0-a75efd97332f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.403908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4643682b-7992-4297-92c0-a75efd97332f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.414312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1334/2692] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:23 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:49:23.418637 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.424152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:23.431506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2730e3ff-a863-4d10-bd31-c0c2e456c50c admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.436500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.437161 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.437441 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.437711 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] PATH_INFO: `/v3/projects/97ac54fbdc6147db8c6cfd5ab9d8cc19/users/e5cd83b9daa649e39219a394bc15d3fd/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.439723 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97ac54fbdc6147db8c6cfd5ab9d8cc19, user_id=e5cd83b9daa649e39219a394bc15d3fd, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.442469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.453907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1359/2693] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:23 2022] PUT /identity/v3/projects/97ac54fbdc6147db8c6cfd5ab9d8cc19/users/e5cd83b9daa649e39219a394bc15d3fd/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:23.465038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.465323 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.465598 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.493954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] MFA Rules not processed for user `e5cd83b9daa649e39219a394bc15d3fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:23.495408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:23.566095 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1335/2694] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:23.578169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.579979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:23.639715 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25651409-06d5-4904-8f36-52dcc0fe1184 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.644573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.645364 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.645677 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.645955 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.655627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.657056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.666499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1360/2695] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:23 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:23.746790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:23.757467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f8865a-1224-49ef-8491-ce1e74b532f4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:23.762970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b06c4e0-b995-42ef-89b3-c50785223ce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:23.763777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06c4e0-b995-42ef-89b3-c50785223ce0 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:23.764267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06c4e0-b995-42ef-89b3-c50785223ce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:23.764651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b06c4e0-b995-42ef-89b3-c50785223ce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:23.841111 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06c4e0-b995-42ef-89b3-c50785223ce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:23.842614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b06c4e0-b995-42ef-89b3-c50785223ce0 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:23.853307 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1336/2696] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:26.619944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:26.687451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80505dc6-f33e-4cfd-ac89-86e57a933d75 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:26.692165 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:26.692862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:26.693140 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:26.693416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] PATH_INFO: `/v3/users/7c5129b1ac54492984a5d0321b0fedec` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:26.695326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c5129b1ac54492984a5d0321b0fedec)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:26.697522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:26.748175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c5129b1ac54492984a5d0321b0fedec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:26.748876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c5129b1ac54492984a5d0321b0fedec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:26.749481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c5129b1ac54492984a5d0321b0fedec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:26.756909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c5129b1ac54492984a5d0321b0fedec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:26.770589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c5129b1ac54492984a5d0321b0fedec'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:26.783181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1361/2697] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:26 2022] DELETE /identity/v3/users/7c5129b1ac54492984a5d0321b0fedec => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:26.787710 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:26.788253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:26.788440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:26.823519 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:26.824737 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:26.865055 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:26.889815 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1337/2698] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:49:26.899423 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:26.900853 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:26.922730 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:26.972004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c908cb40-5de8-4474-8080-bdf6a603204f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:26.976904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:26.977649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:26.977931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:26.978212 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:26.988119 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:26.989638 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:27.000510 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1362/2699] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:27.301227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:27.307449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:27.326310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f8174-c005-4044-a7f1-ae3518fb7fce None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:27.334681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:27.334681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:27.334946 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:27.335727 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] PATH_INFO: `/v3/projects/f447662b94ea464790994cb233d8a16f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:27.336931 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f447662b94ea464790994cb233d8a16f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:27.338979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:27.390237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f447662b94ea464790994cb233d8a16f', 'request_id': 'req-4bb16a70-441a-47e7-9ea7-8609938fbb47'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.395672 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f447662b94ea464790994cb233d8a16f', 'request_id': 'req-4bb16a70-441a-47e7-9ea7-8609938fbb47'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.403831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f447662b94ea464790994cb233d8a16f', 'request_id': 'req-4bb16a70-441a-47e7-9ea7-8609938fbb47'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.415845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] The token cache is being invalidate because project f447662b94ea464790994cb233d8a16f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:27.416132 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.416773 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.418282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1338/2700] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:27 2022] DELETE /identity/v3/projects/f447662b94ea464790994cb233d8a16f => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:27.422323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:27.432217 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933bb76c-a296-4a58-a909-6c8edb4b9a92 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:27.488931 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:27.489570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:27.489850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:27.490125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] PATH_INFO: `/v3/users/e5cd83b9daa649e39219a394bc15d3fd` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:27.491326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5cd83b9daa649e39219a394bc15d3fd)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:27.493257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:27.548686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5cd83b9daa649e39219a394bc15d3fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.549703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5cd83b9daa649e39219a394bc15d3fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.550608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5cd83b9daa649e39219a394bc15d3fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.560480 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5cd83b9daa649e39219a394bc15d3fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.578944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-05622939-7f70-47ca-8431-49d537beca8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5cd83b9daa649e39219a394bc15d3fd'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:27.592705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1363/2701] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:49:27 2022] DELETE /identity/v3/users/e5cd83b9daa649e39219a394bc15d3fd => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:27.887942 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:27.889441 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:27.912721 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb16a70-441a-47e7-9ea7-8609938fbb47 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:27.989565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:27.990205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:27.990483 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:27.990764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] PATH_INFO: `/v3/projects/97ac54fbdc6147db8c6cfd5ab9d8cc19` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:27.992523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97ac54fbdc6147db8c6cfd5ab9d8cc19)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:27.994466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.042608 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97ac54fbdc6147db8c6cfd5ab9d8cc19', 'request_id': 'req-22b0d6ad-ee69-4b66-b33d-52919d14c273'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:28.048140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97ac54fbdc6147db8c6cfd5ab9d8cc19', 'request_id': 'req-22b0d6ad-ee69-4b66-b33d-52919d14c273'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:28.056565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97ac54fbdc6147db8c6cfd5ab9d8cc19', 'request_id': 'req-22b0d6ad-ee69-4b66-b33d-52919d14c273'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:28.064281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:28.072061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] The token cache is being invalidate because project 97ac54fbdc6147db8c6cfd5ab9d8cc19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:49:28.072527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:28.073673 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-22b0d6ad-ee69-4b66-b33d-52919d14c273 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:49:28.075538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1339/2702] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:49:27 2022] DELETE /identity/v3/projects/97ac54fbdc6147db8c6cfd5ab9d8cc19 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:28.082143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.082688 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.082688 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.103654 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:28.105191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:28.134112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:28.166175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1364/2703] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:28.176647 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.177342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.182367 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.204434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:28.205649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:28.214092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1340/2704] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:28.218445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.228975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe34f2d-6821-40a1-9e30-0f2d5f476b8b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.233652 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.234287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.234572 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.234846 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.235493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:49:28.235796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.237387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.249156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1365/2705] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:49:28.253697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.263139 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855eacdc-4128-4f82-96b5-1f70549e17a3 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.268015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.268613 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.268892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.269161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.269794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.271741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.300901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1341/2706] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:28.305334 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.315827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5082f42-8f34-4b32-9e06-5ea90cb45711 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.373389 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.374007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.374394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.374672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.375310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.377542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.384846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.385177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:28.408328 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1366/2707] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:28.412509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.424113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ec5ec1-c54b-4c9d-9d5b-48b0b9fe57de admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.429337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.429997 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.430420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.430722 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.431353 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.432957 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.447113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1367/2708] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:49:28.452009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.462257 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10223d2f-b47e-4456-9b48-bed92697e8b6 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.467299 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.467909 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.468182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.468442 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.469123 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.471070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.481630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1368/2709] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:28.486717 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.488564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c5c9a-3685-424d-8e80-a3c042ae712d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.493916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.494566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.494848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.495119 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.496984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:28.497543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d260b-88fb-4317-bfb4-0dd42dd14b90 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.502261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.502922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.503227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.503520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] PATH_INFO: `/v3/projects/8559f42c708945469875348aed10bf42/users/97f07574df4f4f6cba339d9fe48c29b6/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.509795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.511443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe16962-0d19-47f6-b588-7d707280f87b service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.519754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8559f42c708945469875348aed10bf42, user_id=97f07574df4f4f6cba339d9fe48c29b6, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.522022 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1342/2710] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:28.522496 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.542337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1369/2711] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:28 2022] PUT /identity/v3/projects/8559f42c708945469875348aed10bf42/users/97f07574df4f4f6cba339d9fe48c29b6/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:49:28.554049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.554414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.554631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.589478 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] MFA Rules not processed for user `97f07574df4f4f6cba339d9fe48c29b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:28.592892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:28.672472 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1343/2712] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:49:28.677320 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.750341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcd87d-09eb-401f-ad0f-a09d76eaa594 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.757802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.758607 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.758892 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.759164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.759849 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.761754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.772939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:28.796169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1370/2713] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:49:28.805857 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.825082 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4381d27d-e726-400d-be77-4ffbcc200974 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.880078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:28.900404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.900404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.900404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.900404 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.901563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.902527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.926852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1344/2714] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:49:28 2022] POST /identity/v3/users => generated 295 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:49:28.931633 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.942457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9528fc1d-d4a3-4ac7-8843-db6ae07ffefd admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.948407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.949027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.949303 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.949576 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:28.950259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:28.951882 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:28.970827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1371/2715] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:49:28.975698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:28.986688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af32ea-a974-4052-b482-181ff429a6e3 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:28.991362 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:28.992584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:28.992855 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:28.993148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] PATH_INFO: `/v3/projects/5122270fac824054ad8f60b6b6032d39/users/f676f18e8f5e45d5959de670dc97d206/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:29.014888 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5122270fac824054ad8f60b6b6032d39, user_id=f676f18e8f5e45d5959de670dc97d206, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:29.017528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:29.030930 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1345/2716] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:28 2022] PUT /identity/v3/projects/5122270fac824054ad8f60b6b6032d39/users/f676f18e8f5e45d5959de670dc97d206/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:29.034952 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:29.046014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244813-8423-4e4a-8b41-d7675d668044 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:29.119963 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7619d18f-5981-40b4-aae1-fa437d8e7147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:29.120915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619d18f-5981-40b4-aae1-fa437d8e7147 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:29.120915 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619d18f-5981-40b4-aae1-fa437d8e7147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:29.121310 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7619d18f-5981-40b4-aae1-fa437d8e7147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:29.121773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619d18f-5981-40b4-aae1-fa437d8e7147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:29.123408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7619d18f-5981-40b4-aae1-fa437d8e7147 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:29.134697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1372/2717] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:49:29 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:49:29.139872 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:29.152162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056eee19-9214-443f-9f83-c6b02200e09d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:29.157305 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:29.157963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:29.158301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:29.158538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] PATH_INFO: `/v3/projects/5122270fac824054ad8f60b6b6032d39/users/f676f18e8f5e45d5959de670dc97d206/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:29.160466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5122270fac824054ad8f60b6b6032d39, user_id=f676f18e8f5e45d5959de670dc97d206, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:29.163134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:29.173973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1346/2718] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:49:29 2022] PUT /identity/v3/projects/5122270fac824054ad8f60b6b6032d39/users/f676f18e8f5e45d5959de670dc97d206/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:49:29.186026 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:29.186427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:29.186862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:29.219358 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] MFA Rules not processed for user `f676f18e8f5e45d5959de670dc97d206`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:49:29.220591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:29.299399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1373/2719] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:49:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:49:29.310651 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:29.312670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:49:29.389852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3990431f-d5d3-4e4b-bcf5-f7db72aad40e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:29.393699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:29.394288 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:29.394582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:29.394858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:29.456400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:29.457754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:29.467553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1347/2720] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:37.224969 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:49:37.234555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c7c35c-7f05-4e22-9b4e-dfde85a98584 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:49:37.238113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13849dbb-afcc-43c6-bdec-504170e86aee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:49:37.238743 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849dbb-afcc-43c6-bdec-504170e86aee service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:49:37.239020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849dbb-afcc-43c6-bdec-504170e86aee service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:49:37.239292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13849dbb-afcc-43c6-bdec-504170e86aee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:49:37.248866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849dbb-afcc-43c6-bdec-504170e86aee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:49:37.250314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13849dbb-afcc-43c6-bdec-504170e86aee service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:49:37.260418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1374/2721] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:49:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:49:39.463471 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=70,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:49:47.255365 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=78,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:21.768553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:21.834520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13aa2b5d-f3cf-47ba-a008-d2496d7e20d8 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:21.839068 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:21.839700 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:21.839978 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:21.840250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] PATH_INFO: `/v3/users/97f07574df4f4f6cba339d9fe48c29b6` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:21.841948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f07574df4f4f6cba339d9fe48c29b6)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:21.843958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:21.889364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f07574df4f4f6cba339d9fe48c29b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:21.890012 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f07574df4f4f6cba339d9fe48c29b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:21.890661 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f07574df4f4f6cba339d9fe48c29b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:21.897601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f07574df4f4f6cba339d9fe48c29b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:21.905979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f07574df4f4f6cba339d9fe48c29b6'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:21.923283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1348/2722] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:50:21 2022] DELETE /identity/v3/users/97f07574df4f4f6cba339d9fe48c29b6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:21.928006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:21.928511 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:21.928620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:21.946888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:21.948078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:22.005940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1375/2723] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:50:22.016377 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:22.017862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:22.075956 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1db3eb8-c2be-42ad-9d92-0e149560fb66 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:22.079631 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:22.080240 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:22.080526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:22.080799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:22.090099 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:22.091588 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:22.101548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1349/2724] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:22.408109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:22.410805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:22.424948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00cfa6-eac1-4938-80af-ea5e4d51814a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:22.435799 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:22.436449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:22.436728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:22.437001 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] PATH_INFO: `/v3/projects/8559f42c708945469875348aed10bf42` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:22.438136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8559f42c708945469875348aed10bf42)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:22.440084 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:22.490250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8559f42c708945469875348aed10bf42', 'request_id': 'req-8c356272-ab2c-42bb-b395-83043396f9b8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.495702 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8559f42c708945469875348aed10bf42', 'request_id': 'req-8c356272-ab2c-42bb-b395-83043396f9b8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.502449 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8559f42c708945469875348aed10bf42', 'request_id': 'req-8c356272-ab2c-42bb-b395-83043396f9b8'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.515874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] The token cache is being invalidate because project 8559f42c708945469875348aed10bf42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:50:22.516096 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.516737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.518420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1376/2725] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:50:22 2022] DELETE /identity/v3/projects/8559f42c708945469875348aed10bf42 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:22.522229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:22.531880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd240f87-481c-4ff2-8b2d-d542927aa9db service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:22.550263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:22.589582 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:22.590253 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:22.590528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:22.590839 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] PATH_INFO: `/v3/users/f676f18e8f5e45d5959de670dc97d206` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:22.591943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f676f18e8f5e45d5959de670dc97d206)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:22.593856 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:22.639284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f676f18e8f5e45d5959de670dc97d206'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.639935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f676f18e8f5e45d5959de670dc97d206'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.640538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f676f18e8f5e45d5959de670dc97d206'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.647208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f676f18e8f5e45d5959de670dc97d206'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.654274 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-9710445c-407d-4cdb-ae1a-5b8fd8762653 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f676f18e8f5e45d5959de670dc97d206'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:22.671379 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1350/2726] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:50:22 2022] DELETE /identity/v3/users/f676f18e8f5e45d5959de670dc97d206 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:22.943631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:22.945081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:22.958552 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c356272-ab2c-42bb-b395-83043396f9b8 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.015847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.016507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.016786 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.017056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] PATH_INFO: `/v3/projects/5122270fac824054ad8f60b6b6032d39` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.018429 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5122270fac824054ad8f60b6b6032d39)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.020258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.051512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:23.074985 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5122270fac824054ad8f60b6b6032d39', 'request_id': 'req-6edef4ad-bd76-488b-9813-4d103045e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:23.084805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5122270fac824054ad8f60b6b6032d39', 'request_id': 'req-6edef4ad-bd76-488b-9813-4d103045e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:23.091502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5122270fac824054ad8f60b6b6032d39', 'request_id': 'req-6edef4ad-bd76-488b-9813-4d103045e5d4'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:23.108102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] The token cache is being invalidate because project 5122270fac824054ad8f60b6b6032d39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:50:23.108392 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:23.109037 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-6edef4ad-bd76-488b-9813-4d103045e5d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:23.110554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1377/2727] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:50:22 2022] DELETE /identity/v3/projects/5122270fac824054ad8f60b6b6032d39 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:23.116418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.116831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.116948 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.136935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:23.138235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:23.195912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1351/2728] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:23.259521 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.259803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.260104 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.279452 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:23.280592 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:23.288570 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1378/2729] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:23.292999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.303490 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b42ca3-b239-4d5e-9e03-46d4dcbdb1ee None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.307985 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.308584 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.308859 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.309155 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.310459 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:50:23.310761 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.312341 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.323846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1352/2730] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 24 03:50:23.328507 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.337720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e55522b-79c3-4400-9c3d-9a0b72835c51 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.342487 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.343078 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.343356 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.343626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.344254 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.346134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.374192 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1379/2731] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:50:23 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:50:23.378313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.388180 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e2c304-d5d7-47d7-a274-954548b2467e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.413729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:23.446513 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.447111 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.447394 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.447662 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.448281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.450597 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.471217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1353/2732] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:50:23 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:50:23.475408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.486155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b9e776-adc8-42fe-8d61-bf008dd832ae admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.491057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.491666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.491941 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.492322 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.492922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.494454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.505107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1380/2733] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:50:23.509130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.520258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a506a2a-9676-4cec-a49c-adc158f4b07d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.524984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd7673c-c846-4cf8-a6bd-615849a8f5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.525668 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7673c-c846-4cf8-a6bd-615849a8f5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.525952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7673c-c846-4cf8-a6bd-615849a8f5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.526260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd7673c-c846-4cf8-a6bd-615849a8f5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.526932 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7673c-c846-4cf8-a6bd-615849a8f5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.528666 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd7673c-c846-4cf8-a6bd-615849a8f5e8 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.543751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1354/2734] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:50:23.545010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.555357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbae412-6e04-4eed-b867-b08741e4c005 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.560696 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.561346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.561740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.562023 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] PATH_INFO: `/v3/projects/00feec1c522248d68812d9f6945eff8d/users/51db9bf57a6b4d2bbc64143187d9be8f/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.578043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] RBAC: Authorizing `identity:create_grant(project_id=00feec1c522248d68812d9f6945eff8d, user_id=51db9bf57a6b4d2bbc64143187d9be8f, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.580778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.596231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1381/2735] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:50:23 2022] PUT /identity/v3/projects/00feec1c522248d68812d9f6945eff8d/users/51db9bf57a6b4d2bbc64143187d9be8f/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:23.602260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4cbf-644f-49c0-a398-531c0e0f1713 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.602804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4cbf-644f-49c0-a398-531c0e0f1713 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.603176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e4cbf-644f-49c0-a398-531c0e0f1713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.621563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-bf6e4cbf-644f-49c0-a398-531c0e0f1713 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:23.623314 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bf6e4cbf-644f-49c0-a398-531c0e0f1713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:23.681836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1355/2736] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:23.692166 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:23.693948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:23.753514 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14cd4fb-3e24-4a69-bcf6-fe0d7a2114af admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:23.757179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:23.757785 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:23.758069 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:23.758381 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:23.768050 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:23.769501 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:23.778923 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1382/2737] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:24.394992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.395527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.395527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.423372 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] MFA Rules not processed for user `51db9bf57a6b4d2bbc64143187d9be8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:24.424585 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:24.494544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1356/2738] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:24.499048 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:24.508477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a336cfde-a2ee-44cd-8aab-1a79e1d3ab18 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:24.513181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:24.513790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.514073 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.514377 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.515028 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:24.516932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:24.545850 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1383/2739] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:50:24 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:50:24.549875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:24.560949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae4663-d7f5-4c7d-af4f-e28f612621fb None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:24.596997 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:24.625366 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:24.625980 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.626283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.626559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.627369 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:24.629744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:24.652335 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1357/2740] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:50:24 2022] POST /identity/v3/users => generated 292 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:50:24.656780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:24.668238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbae46fb-dcff-4b8e-b720-579f2f2b8640 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:24.675009 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:24.675685 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.675968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.676241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.676894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:24.678651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:24.691198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1384/2741] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:50:24.696045 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:24.706878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bddc00-9f45-4447-9dbe-93fbe021b29a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:24.713620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:24.714322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.714603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.714878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] PATH_INFO: `/v3/projects/3d6efaed52b344d7aca73e4ef52aeedf/users/79b0af505234413f9c6b43e41fe879bb/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.735664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6efaed52b344d7aca73e4ef52aeedf, user_id=79b0af505234413f9c6b43e41fe879bb, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:24.738786 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:24.757016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1358/2742] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:50:24 2022] PUT /identity/v3/projects/3d6efaed52b344d7aca73e4ef52aeedf/users/79b0af505234413f9c6b43e41fe879bb/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:24.761563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:24.773980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf0aa-4064-4efc-82b7-4fdd6241f843 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:24.795700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:24.847163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:24.847163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.847163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.847163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.847581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:24.847581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:24.858947 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1385/2743] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:50:24.860309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:24.870709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf6e6ba-c9c8-4b7d-9e93-4f6861ebddb6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:24.875302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334cc01b-82b6-4359-9b0b-8be8cc62def9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:24.876071 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc01b-82b6-4359-9b0b-8be8cc62def9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:24.876349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc01b-82b6-4359-9b0b-8be8cc62def9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:24.876620 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334cc01b-82b6-4359-9b0b-8be8cc62def9 admin admin] PATH_INFO: `/v3/projects/3d6efaed52b344d7aca73e4ef52aeedf/users/79b0af505234413f9c6b43e41fe879bb/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:24.878698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cc01b-82b6-4359-9b0b-8be8cc62def9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d6efaed52b344d7aca73e4ef52aeedf, user_id=79b0af505234413f9c6b43e41fe879bb, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:24.881336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334cc01b-82b6-4359-9b0b-8be8cc62def9 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:24.893466 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1359/2744] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:50:24 2022] PUT /identity/v3/projects/3d6efaed52b344d7aca73e4ef52aeedf/users/79b0af505234413f9c6b43e41fe879bb/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:25.225360 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:25.227508 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:25.325109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9116b-d27b-4567-b909-16c1f03dfe9e admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:25.411571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:25.412623 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:25.412899 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:25.413178 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:25.415754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:25.428839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:25.430621 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:25.442481 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1386/2745] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:25.739136 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:25.739642 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:25.739753 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:25.767327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] MFA Rules not processed for user `79b0af505234413f9c6b43e41fe879bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:25.768551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:25.854326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1360/2746] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:25 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:25.864148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:25.865670 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:25.938227 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=100 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:25.954436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754a8b3f-9ad6-4676-85cd-7cc5550f25da service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:25.959693 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5056f30-5d37-4493-9f0d-2aef424a1cec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:25.960566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056f30-5d37-4493-9f0d-2aef424a1cec service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:25.961152 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056f30-5d37-4493-9f0d-2aef424a1cec service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:25.961551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5056f30-5d37-4493-9f0d-2aef424a1cec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:26.032234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5056f30-5d37-4493-9f0d-2aef424a1cec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:26.033745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5056f30-5d37-4493-9f0d-2aef424a1cec service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:26.042922 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1387/2747] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:27.395707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:27.397244 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:27.409079 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abac942-2355-417c-9709-e4270f9edea2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:27.412612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:27.413205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:27.413484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:27.413755 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:27.422660 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:27.424084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:27.433415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1361/2748] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:27 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:36.038801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=30 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:37.429287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:37.944319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:37.944811 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:37.944913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:37.964105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] MFA Rules not processed for user `bec33f84fef94b56ab0e1639bb8f9b14`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:37.965847 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:38.030090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1388/2749] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:37 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:38.312831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:38.370825 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c21e26-a67d-43a2-af4d-a1387f40413c service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:38.374420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:38.375021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:38.375298 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:38.375569 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:38.377326 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:38.388228 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:38.389678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:38.397896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1362/2750] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:50:46.927292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:46.928739 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:46.985690 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:47.006800 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6038ba-b80c-4b88-8ca8-68d44b0004be None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:47.012041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:47.012649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:47.012926 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:47.013197 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:47.015024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:47.027602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:47.029189 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:47.040944 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1389/2751] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:47.060868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d92445-2323-456b-94ab-0c37f6a917cf service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:47.066067 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:47.066875 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:47.067279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:47.068138 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] PATH_INFO: `/v3/users/51db9bf57a6b4d2bbc64143187d9be8f` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:47.069951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] RBAC: Authorizing `identity:delete_user(user_id=51db9bf57a6b4d2bbc64143187d9be8f)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:47.072731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:47.123681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51db9bf57a6b4d2bbc64143187d9be8f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.124556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51db9bf57a6b4d2bbc64143187d9be8f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.125168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51db9bf57a6b4d2bbc64143187d9be8f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.132230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51db9bf57a6b4d2bbc64143187d9be8f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.140411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51db9bf57a6b4d2bbc64143187d9be8f'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.158898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1363/2752] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:50:46 2022] DELETE /identity/v3/users/51db9bf57a6b4d2bbc64143187d9be8f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:50:47.336876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:47.363861 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2529a8f0-91a2-4023-bc5f-a6d4ed8a8bd7 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:47.442563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:47.443287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:47.443698 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:47.443971 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] PATH_INFO: `/v3/projects/00feec1c522248d68812d9f6945eff8d` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:47.445132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] RBAC: Authorizing `identity:delete_project(project_id=00feec1c522248d68812d9f6945eff8d)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:47.447097 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:47.489164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:47.499617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00feec1c522248d68812d9f6945eff8d', 'request_id': 'req-d8b93bec-410b-42df-8eee-076b151e039b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.505464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00feec1c522248d68812d9f6945eff8d', 'request_id': 'req-d8b93bec-410b-42df-8eee-076b151e039b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.518680 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00feec1c522248d68812d9f6945eff8d', 'request_id': 'req-d8b93bec-410b-42df-8eee-076b151e039b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.533109 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] The token cache is being invalidate because project 00feec1c522248d68812d9f6945eff8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:50:47.533409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.534258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.536173 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1390/2753] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:50:47 2022] DELETE /identity/v3/projects/00feec1c522248d68812d9f6945eff8d => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:47.541260 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:47.543084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:47.560073 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4477269b-97c7-4cd7-9401-202e1f8d210f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:47.579515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:47.628770 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:47.629451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:47.629835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:47.630226 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] PATH_INFO: `/v3/users/79b0af505234413f9c6b43e41fe879bb` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:47.631556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79b0af505234413f9c6b43e41fe879bb)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:47.634824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:47.692763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79b0af505234413f9c6b43e41fe879bb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.693816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79b0af505234413f9c6b43e41fe879bb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.694776 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79b0af505234413f9c6b43e41fe879bb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.707965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79b0af505234413f9c6b43e41fe879bb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.719183 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-aabfaa31-5a91-4422-8857-721921dfaa63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79b0af505234413f9c6b43e41fe879bb'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:47.744354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1364/2754] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:50:47 2022] DELETE /identity/v3/users/79b0af505234413f9c6b43e41fe879bb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:47.952613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:47.954705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:47.969888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b93bec-410b-42df-8eee-076b151e039b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.040719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.041354 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.041631 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.041901 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] PATH_INFO: `/v3/projects/3d6efaed52b344d7aca73e4ef52aeedf` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.043554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d6efaed52b344d7aca73e4ef52aeedf)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.045489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.100228 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d6efaed52b344d7aca73e4ef52aeedf', 'request_id': 'req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:48.106830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d6efaed52b344d7aca73e4ef52aeedf', 'request_id': 'req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:48.113695 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d6efaed52b344d7aca73e4ef52aeedf', 'request_id': 'req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:48.128086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] The token cache is being invalidate because project 3d6efaed52b344d7aca73e4ef52aeedf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:50:48.128420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:48.129402 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-7b320eb7-47e9-4ae1-ae5c-33e9b8981927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:48.132007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1391/2755] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:50:47 2022] DELETE /identity/v3/projects/3d6efaed52b344d7aca73e4ef52aeedf => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:48.137049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.137538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.137665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.157495 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:48.158694 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:48.225388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1365/2756] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:48.233900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.234200 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.234491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.254062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:48.255313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:48.269038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1392/2757] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:50:48.273605 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.283923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1859657-4fb5-4a3c-b1ab-5c68af2aa644 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.288595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.289187 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.289465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.289729 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.290407 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:50:48.290708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.292923 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.305004 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1366/2758] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:50:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:50:48.309566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.318888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9b4657-2f9e-4fd1-b26e-8d37988a38c0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.323889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.324503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.324778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.325047 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.325665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.327574 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.363752 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1393/2759] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:50:48.370617 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.383171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3427ff55-2d38-4e6c-80d4-fb1a1231f52a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.409148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:48.442665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.443265 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.443556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.443835 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.444456 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.446399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.468283 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1367/2760] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:50:48.472725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.483435 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac56ddae-3ec2-4941-927a-eb37d20d4e44 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.488371 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.488968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.489245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.489522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.490204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.491777 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.502416 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1394/2761] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:50:48.506908 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.519239 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0ac41f-4879-47b6-9994-ba86d8e22508 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.526320 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88505207-ed07-43dc-bf6f-9ca21cc83928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.527708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505207-ed07-43dc-bf6f-9ca21cc83928 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.528211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505207-ed07-43dc-bf6f-9ca21cc83928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.528758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88505207-ed07-43dc-bf6f-9ca21cc83928 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.529971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88505207-ed07-43dc-bf6f-9ca21cc83928 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.533174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88505207-ed07-43dc-bf6f-9ca21cc83928 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.552522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1368/2762] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:50:48.557964 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.569934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83647a6d-21f9-46c5-9a36-993dc9162cba admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.575626 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.576461 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.576713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.577038 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] PATH_INFO: `/v3/projects/1cfb49807ba94a0790770ad0262487a1/users/10817c7a8f354a08a98971f5567eb469/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.595203 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:48.603939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cfb49807ba94a0790770ad0262487a1, user_id=10817c7a8f354a08a98971f5567eb469, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.607246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.627388 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1395/2763] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:50:48 2022] PUT /identity/v3/projects/1cfb49807ba94a0790770ad0262487a1/users/10817c7a8f354a08a98971f5567eb469/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:48.638969 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.639159 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.639391 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.670809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] MFA Rules not processed for user `10817c7a8f354a08a98971f5567eb469`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:48.671941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:48.741935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1369/2764] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:48.746807 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.814594 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a710c963-98ac-4b72-affe-bc4f80de4a28 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.819259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.819851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.820128 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.820397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.821014 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.822927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.853124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1396/2765] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:50:48.857558 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.868782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc2061f-629f-4066-9343-37e4b91a6bf4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.925211 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.925731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.926010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.926322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.926961 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.928890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.953973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1370/2766] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:50:48 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:50:48.957956 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:48.969246 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c7231d-c138-4e2b-a792-74bb932a04d3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:48.973973 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:48.974608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:48.974891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:48.975164 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:48.975803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:48.977385 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:48.992520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1397/2767] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:50:48.997242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:49.007571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45e33c6-dce9-4bc6-a591-3a7b6ff9b9b6 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:49.013599 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:49.014364 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:49.014686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:49.015035 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] PATH_INFO: `/v3/projects/38c5a767a1754d929f45daabca1dc939/users/cb783f39227a464b9a9d6b3400115531/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:49.030655 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c5a767a1754d929f45daabca1dc939, user_id=cb783f39227a464b9a9d6b3400115531, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:49.033350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:49.047143 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1371/2768] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:50:48 2022] PUT /identity/v3/projects/38c5a767a1754d929f45daabca1dc939/users/cb783f39227a464b9a9d6b3400115531/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:49.052577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:49.063391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f7b8f-028d-47b1-8e23-23710676ba7a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:49.126258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fc744-7222-45f2-8e15-5d96c34df0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:49.127086 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fc744-7222-45f2-8e15-5d96c34df0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:49.127409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fc744-7222-45f2-8e15-5d96c34df0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:49.127748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fc744-7222-45f2-8e15-5d96c34df0f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:49.128593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fc744-7222-45f2-8e15-5d96c34df0f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:49.130321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fc744-7222-45f2-8e15-5d96c34df0f7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:49.140920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1398/2769] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:50:49 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:50:49.145596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:49.151094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:49.157862 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e274c-b3fc-4ba2-b26e-a8cf7645744a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:49.163987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:49.164790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:49.165182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:49.165182 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] PATH_INFO: `/v3/projects/38c5a767a1754d929f45daabca1dc939/users/cb783f39227a464b9a9d6b3400115531/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:49.167261 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c5a767a1754d929f45daabca1dc939, user_id=cb783f39227a464b9a9d6b3400115531, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:49.169878 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:49.181178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1372/2770] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:50:49 2022] PUT /identity/v3/projects/38c5a767a1754d929f45daabca1dc939/users/cb783f39227a464b9a9d6b3400115531/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:49.193125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:49.193434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:49.193710 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:49.221954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] MFA Rules not processed for user `cb783f39227a464b9a9d6b3400115531`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:49.223196 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:49.270314 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:49.291399 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1399/2771] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:49 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:50:49.302349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:49.304137 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:49.374999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbff486-71c4-4b87-a6eb-f2dac616a620 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:49.378665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:49.379266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:49.379542 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:49.379809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:49.438863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:49.440230 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:49.449606 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1373/2772] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 24 03:50:53.803146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:53.804627 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:53.817186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a1c5b-9423-495f-96a8-e3633a1431fb None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:53.820796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9460fa5-25d1-4855-b579-d6eb6557ff63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:53.821394 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460fa5-25d1-4855-b579-d6eb6557ff63 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:53.821669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460fa5-25d1-4855-b579-d6eb6557ff63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:53.821937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9460fa5-25d1-4855-b579-d6eb6557ff63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:53.831708 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9460fa5-25d1-4855-b579-d6eb6557ff63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:53.833120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9460fa5-25d1-4855-b579-d6eb6557ff63 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:53.843121 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1400/2773] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:53 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:58.837987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:58.905841 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65be2d2-e1c4-4a62-b1eb-0b5f247bd175 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:58.910445 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:58.911078 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:58.911357 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:58.911626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] PATH_INFO: `/v3/users/10817c7a8f354a08a98971f5567eb469` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:58.912751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10817c7a8f354a08a98971f5567eb469)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:58.914699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:58.965189 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:58.967981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10817c7a8f354a08a98971f5567eb469'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:58.968975 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10817c7a8f354a08a98971f5567eb469'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:58.969836 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10817c7a8f354a08a98971f5567eb469'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:58.981077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10817c7a8f354a08a98971f5567eb469'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:58.989575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10817c7a8f354a08a98971f5567eb469'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.012346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1374/2774] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:50:58 2022] DELETE /identity/v3/users/10817c7a8f354a08a98971f5567eb469 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:50:59.020051 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:59.020781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:59.020961 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:59.045213 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:50:59.046489 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:59.107277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1401/2775] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:50:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:50:59.118869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:59.120966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:59.180148 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7027bdfc-7437-4bba-8da3-ac09a33c9609 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:59.184356 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:59.184949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:59.185353 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:59.185611 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:59.194809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:59.196243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:59.206991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1375/2776] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:50:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:50:59.479090 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:59.481139 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:50:59.496821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f81cdd-de45-4b9d-856a-6b04a40d413c None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:59.501545 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:59.502207 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:59.502486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:59.502759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] PATH_INFO: `/v3/projects/1cfb49807ba94a0790770ad0262487a1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:59.503884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cfb49807ba94a0790770ad0262487a1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:59.505801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:59.564999 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cfb49807ba94a0790770ad0262487a1', 'request_id': 'req-57e0977a-c568-4a7b-960b-85fb2f795541'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.570869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cfb49807ba94a0790770ad0262487a1', 'request_id': 'req-57e0977a-c568-4a7b-960b-85fb2f795541'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.578913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cfb49807ba94a0790770ad0262487a1', 'request_id': 'req-57e0977a-c568-4a7b-960b-85fb2f795541'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.592174 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] The token cache is being invalidate because project 1cfb49807ba94a0790770ad0262487a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:50:59.592460 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.593082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.594984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1402/2777] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:50:59 2022] DELETE /identity/v3/projects/1cfb49807ba94a0790770ad0262487a1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:50:59.598760 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:50:59.609339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ed3bf-6e1b-4029-9a37-e64e4438b251 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:50:59.668795 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:50:59.669418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:50:59.669698 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:50:59.669972 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] PATH_INFO: `/v3/users/cb783f39227a464b9a9d6b3400115531` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:50:59.671135 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb783f39227a464b9a9d6b3400115531)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:50:59.673056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:50:59.724306 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb783f39227a464b9a9d6b3400115531'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.724877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb783f39227a464b9a9d6b3400115531'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.725475 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb783f39227a464b9a9d6b3400115531'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.733278 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb783f39227a464b9a9d6b3400115531'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.743164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb783f39227a464b9a9d6b3400115531'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:50:59.751999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:50:59.770965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1376/2778] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:50:59 2022] DELETE /identity/v3/users/cb783f39227a464b9a9d6b3400115531 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:00.100744 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:00.102268 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:00.115502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0977a-c568-4a7b-960b-85fb2f795541 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:00.175803 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:00.182783 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:00.184060 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:00.184350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:00.184874 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] PATH_INFO: `/v3/projects/38c5a767a1754d929f45daabca1dc939` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:00.186646 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c5a767a1754d929f45daabca1dc939)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:00.189948 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:00.238943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c5a767a1754d929f45daabca1dc939', 'request_id': 'req-186cdca6-5046-4cba-b8db-dc54d0f61169'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:00.244866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c5a767a1754d929f45daabca1dc939', 'request_id': 'req-186cdca6-5046-4cba-b8db-dc54d0f61169'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:00.251793 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c5a767a1754d929f45daabca1dc939', 'request_id': 'req-186cdca6-5046-4cba-b8db-dc54d0f61169'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:00.265645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] The token cache is being invalidate because project 38c5a767a1754d929f45daabca1dc939 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:51:00.265645 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:00.266154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-186cdca6-5046-4cba-b8db-dc54d0f61169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:00.267700 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1403/2779] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:51:00 2022] DELETE /identity/v3/projects/38c5a767a1754d929f45daabca1dc939 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:03.320562 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:03.322050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:03.382347 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52343e8-674e-45c4-b17b-6077a1870329 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:03.438451 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:03.439062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:03.439343 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:03.439612 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:03.441409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:03.514977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:03.516437 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:03.525575 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1377/2780] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:03 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:10.261806 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=25,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:13.521405 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=2,keystone:SELECT=73 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:13.948765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:13.949318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:13.949318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:13.970954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] MFA Rules not processed for user `fe0dc1ce495045e4bf158ec503a5b5d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:13.972202 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:13.979120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1404/2781] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:51:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:14.345332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:14.347889 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:14.415281 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6dc221-be71-448c-ab95-c1447eef5dfc service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:14.421834 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:14.422576 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:14.422924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:14.423342 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:14.425452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:14.437160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:14.438664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:14.448179 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1378/2782] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:23.964760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:24.443555 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=26 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:29.518506 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:29.528758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c41cec-4afb-406a-b65e-e5c37bb934dc None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:29.532375 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:29.532977 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:29.533250 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:29.533515 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:29.535276 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:29.607464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:29.608917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:29.623379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1405/2783] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:33.180497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:33.182031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:33.193440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8180c5e-a442-48b6-a4c5-2df58010fb85 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:33.197408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:33.197999 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:33.198303 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:33.198579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:33.200251 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:33.213381 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:33.214843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:33.226279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1379/2784] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:39.615175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=34 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:42.753256 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:42.754801 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:42.819943 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b8088-fa85-4f53-b5e3-2e72c92448a8 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:42.825019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:42.826143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:42.826445 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:42.826712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] PATH_INFO: `/v3/users/bec33f84fef94b56ab0e1639bb8f9b14` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:42.827821 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec33f84fef94b56ab0e1639bb8f9b14)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:42.829705 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:42.876933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec33f84fef94b56ab0e1639bb8f9b14'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:42.877582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec33f84fef94b56ab0e1639bb8f9b14'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:42.878224 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec33f84fef94b56ab0e1639bb8f9b14'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:42.885782 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec33f84fef94b56ab0e1639bb8f9b14'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:42.897477 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-cd7eca9e-d500-47b1-aa79-33403fa8d676 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec33f84fef94b56ab0e1639bb8f9b14'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:42.909543 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1406/2785] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:51:42 2022] DELETE /identity/v3/users/bec33f84fef94b56ab0e1639bb8f9b14 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:43.087110 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.098101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3c8a13-3093-49fc-9196-4a02a5ff0d2f service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.167804 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.168442 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.169258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.169537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] PATH_INFO: `/v3/projects/9b17dad9df1840cc816875bcc42f1706` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.170689 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b17dad9df1840cc816875bcc42f1706)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.172601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.218861 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b17dad9df1840cc816875bcc42f1706', 'request_id': 'req-a42bd843-a44f-4614-93ca-b879ca37edfe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:43.224414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b17dad9df1840cc816875bcc42f1706', 'request_id': 'req-a42bd843-a44f-4614-93ca-b879ca37edfe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:43.230869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b17dad9df1840cc816875bcc42f1706', 'request_id': 'req-a42bd843-a44f-4614-93ca-b879ca37edfe'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:43.242449 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] The token cache is being invalidate because project 9b17dad9df1840cc816875bcc42f1706 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:51:43.242719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:43.243538 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-a42bd843-a44f-4614-93ca-b879ca37edfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:51:43.245198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1380/2786] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:51:43 2022] DELETE /identity/v3/projects/9b17dad9df1840cc816875bcc42f1706 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:43.253040 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.253434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.253642 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.272840 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:43.274053 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:43.333162 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1407/2787] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:51:43.342237 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.342520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.342790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.364108 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:43.365284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:43.377346 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1381/2788] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:51:43.384871 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.394635 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809f5a29-677b-4922-83a2-f2599c574228 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.399159 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.399759 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.400033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.400301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.400933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:51:43.401230 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.402842 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.414994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1408/2789] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:51:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:51:43.420038 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.429218 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aa3f11-fe1b-4d68-8ef3-6ff4065d29f1 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.433913 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.434537 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.434813 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.435081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.435708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.437636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.464797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1382/2790] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:43 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:43.469396 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.479427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f467a42-8fbf-438e-915b-6250e6e20fca admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.483816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:43.537802 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.538434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.538716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.538990 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.539613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.541536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.563624 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1409/2791] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:43 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:43.567828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.579266 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf76258b-5bf4-4f35-b792-8112a908bfa1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.583965 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.584591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.584869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.585142 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.585810 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.587440 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.598248 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1383/2792] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:43.602115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.612651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99450f91-3358-47d0-88c5-fb47f04dfd84 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.617234 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd6bd76-4327-4ac9-a0ab-fac8673ce37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.617831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6bd76-4327-4ac9-a0ab-fac8673ce37f admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.618125 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6bd76-4327-4ac9-a0ab-fac8673ce37f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.618407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd6bd76-4327-4ac9-a0ab-fac8673ce37f admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.619045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6bd76-4327-4ac9-a0ab-fac8673ce37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.620675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd6bd76-4327-4ac9-a0ab-fac8673ce37f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.631589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1410/2793] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:43.635718 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.645516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ed985d-21e4-41d2-93ef-44c9d353134e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.650001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.650665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.650940 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.651206 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] PATH_INFO: `/v3/projects/7b550db4de5841e8a1ffcc7213275499/users/9c1f5f4c1e544708a69b5631af831dd0/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.665322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b550db4de5841e8a1ffcc7213275499, user_id=9c1f5f4c1e544708a69b5631af831dd0, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.667971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.687163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1384/2794] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:43 2022] PUT /identity/v3/projects/7b550db4de5841e8a1ffcc7213275499/users/9c1f5f4c1e544708a69b5631af831dd0/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:51:43.692204 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3242a60-37e9-4511-a7a5-f1e2ce28a192 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.692516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3242a60-37e9-4511-a7a5-f1e2ce28a192 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.692818 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3242a60-37e9-4511-a7a5-f1e2ce28a192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.711451 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-b3242a60-37e9-4511-a7a5-f1e2ce28a192 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:43.712666 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b3242a60-37e9-4511-a7a5-f1e2ce28a192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:43.771584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1411/2795] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:51:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:43.781744 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:43.783467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:43.797432 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:43.843050 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb92c2ac-2209-4d5f-9c04-847692fe1a6c admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:43.846746 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:43.847350 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:43.847635 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:43.847906 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:43.856912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:43.858383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:43.868217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1385/2796] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:51:48.142597 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.143472 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.143932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.177851 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] MFA Rules not processed for user `9c1f5f4c1e544708a69b5631af831dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:48.180124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:48.248619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1412/2797] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:51:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:51:48.253041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:48.263830 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45220fc0-e03a-463d-a065-1ad3c67fddfd service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:48.268573 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:48.269174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.269448 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.269846 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.270526 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:48.272409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:48.311000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1386/2798] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:51:48 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:51:48.315258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:48.327553 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283a3b96-aec3-424b-a4b0-7a0eeada4ee0 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:48.340978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:51:48.390143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:48.390826 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.391105 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.391397 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.392030 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:48.393974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:48.416755 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1413/2799] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:51:48 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:51:48.420780 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:48.431782 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923cc1d8-1b97-458b-9006-e8d662240d32 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:48.437216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:48.437820 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.438124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.438424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.439062 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:48.440657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:48.452592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1387/2800] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:48.456916 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:48.467787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05eea-6c09-4a62-b6b6-5f5a886f733b admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:48.472816 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:48.473483 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.473758 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.474024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] PATH_INFO: `/v3/projects/4abac066f9d94181bbe793fc1e6f38a4/users/d9275b375e664ebfa33098a165751745/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.489737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abac066f9d94181bbe793fc1e6f38a4, user_id=d9275b375e664ebfa33098a165751745, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:48.492408 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:48.506500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1414/2801] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:48 2022] PUT /identity/v3/projects/4abac066f9d94181bbe793fc1e6f38a4/users/d9275b375e664ebfa33098a165751745/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:48.510607 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:48.521017 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019276d3-a152-4260-8780-849244b9e43d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:48.577719 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f177c140-0c91-4984-aa10-1cd3e7b072bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:48.578368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177c140-0c91-4984-aa10-1cd3e7b072bd admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.578650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177c140-0c91-4984-aa10-1cd3e7b072bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.578922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f177c140-0c91-4984-aa10-1cd3e7b072bd admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.579567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177c140-0c91-4984-aa10-1cd3e7b072bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:48.581161 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f177c140-0c91-4984-aa10-1cd3e7b072bd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:48.590883 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1388/2802] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:51:48 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:51:48.595039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:48.605907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c79885-5691-4187-95d5-7e50f58c6503 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:48.610614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:48.611253 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:48.611526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:48.611791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] PATH_INFO: `/v3/projects/4abac066f9d94181bbe793fc1e6f38a4/users/d9275b375e664ebfa33098a165751745/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:48.613669 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4abac066f9d94181bbe793fc1e6f38a4, user_id=d9275b375e664ebfa33098a165751745, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:48.616300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:48.627374 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1415/2803] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:51:48 2022] PUT /identity/v3/projects/4abac066f9d94181bbe793fc1e6f38a4/users/d9275b375e664ebfa33098a165751745/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:51:53.030527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e92dc-31c4-4d93-9c5b-a57acd0e2bf6 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:53.031016 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e92dc-31c4-4d93-9c5b-a57acd0e2bf6 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:53.031522 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e92dc-31c4-4d93-9c5b-a57acd0e2bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:53.065800 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-071e92dc-31c4-4d93-9c5b-a57acd0e2bf6 None None] MFA Rules not processed for user `d9275b375e664ebfa33098a165751745`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:51:53.067081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-071e92dc-31c4-4d93-9c5b-a57acd0e2bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:53.156209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1389/2804] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:51:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:51:53.174132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:51:53.175881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:51:53.294876 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ff37db-0252-4727-8fe7-8c03b9d97f15 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:51:53.303471 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:51:53.304094 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:51:53.304368 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:51:53.304632 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:51:53.365163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:51:53.366880 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:51:53.376838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1416/2805] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:51:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:03.152897 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:03.372290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=5 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:18.239722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:18.240302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:18.240302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:18.258977 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] MFA Rules not processed for user `9c1f5f4c1e544708a69b5631af831dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:52:18.260195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:18.267881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1390/2806] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:52:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:52:20.833885 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:20.835577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.900881 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1ed33d-be97-48de-810c-aa4e40d9f908 service nova] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:20.905044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:20.905663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:20.905938 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:20.906244 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:20.907994 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:52:20.919434 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:20.921012 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:20.930330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1417/2807] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:28.253675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:30.926772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=26 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:52:35.986708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:52:35.996671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5832df96-02e5-4a18-bfbb-3012a4d33c28 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:52:36.000198 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:52:36.000798 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:52:36.001072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:52:36.001338 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:52:36.010498 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:52:36.012570 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:52:36.021881 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1391/2808] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:52:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:52:46.017781 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=6 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:38.974701 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:38.983386 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582a3688-c8fc-411a-ac19-eeacfa75812a service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:38.986917 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:38.987502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:38.987774 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:38.988036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:38.989713 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:39.052651 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:39.054080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:39.064285 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1418/2809] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:42.604284 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:42.605764 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:42.677814 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1376eb-494d-4735-96e4-572169be639d service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:42.683053 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:42.683681 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:42.683963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:42.684241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:42.686075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:42.700424 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:42.702091 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:42.713090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1392/2810] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:42.926525 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:42.928042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:42.941978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce3398f-1652-459c-a518-d8e13d7308f0 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:42.946946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:42.947579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:42.947858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:42.948134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] PATH_INFO: `/v3/users/fe0dc1ce495045e4bf158ec503a5b5d1` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:42.949934 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0dc1ce495045e4bf158ec503a5b5d1)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:42.951891 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:42.997939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0dc1ce495045e4bf158ec503a5b5d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:42.998622 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0dc1ce495045e4bf158ec503a5b5d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:42.999245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0dc1ce495045e4bf158ec503a5b5d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.006904 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0dc1ce495045e4bf158ec503a5b5d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.019690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-e61390a6-5b4f-41b0-bcbf-8e090c957b76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0dc1ce495045e4bf158ec503a5b5d1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.031427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1419/2811] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:42 2022] DELETE /identity/v3/users/fe0dc1ce495045e4bf158ec503a5b5d1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:43.206941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.217754 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c31d76-c16c-41bb-855a-e413e06f582d service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.277987 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.278645 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.278924 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.279201 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] PATH_INFO: `/v3/projects/28b19b24989f449094001356712fc0e0` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.280325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=28b19b24989f449094001356712fc0e0)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.282252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.328028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '28b19b24989f449094001356712fc0e0', 'request_id': 'req-6b9f0f3f-f908-49ec-990f-b908d50aeb40'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.333470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '28b19b24989f449094001356712fc0e0', 'request_id': 'req-6b9f0f3f-f908-49ec-990f-b908d50aeb40'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.341704 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '28b19b24989f449094001356712fc0e0', 'request_id': 'req-6b9f0f3f-f908-49ec-990f-b908d50aeb40'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.354433 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] The token cache is being invalidate because project 28b19b24989f449094001356712fc0e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:53:43.354610 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.355233 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-6b9f0f3f-f908-49ec-990f-b908d50aeb40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:43.357571 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1393/2812] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:53:43 2022] DELETE /identity/v3/projects/28b19b24989f449094001356712fc0e0 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:43.362002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.362176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.362485 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.381748 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:43.383002 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:43.443794 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1420/2813] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:53:43.452165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.452388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.452664 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.474133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:43.475392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:43.483436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1394/2814] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:43.487873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.498340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e6c516-a19e-4d4c-af83-40e3978a1222 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.502980 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.503585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.503866 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.504136 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.504779 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:53:43.505080 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.506707 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.508176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:43.520908 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1421/2815] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:53:43.525090 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.534553 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b0e4f-f43e-4d21-8d60-720bd81b0548 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.539184 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.539775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.540052 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.540325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.540939 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.542890 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.554042 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:43.577732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1395/2816] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:43.581712 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.591237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba934762-1ed7-42fb-86ea-8a7596b2c4e3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.661697 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.662436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.662781 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.663149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.663839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.665762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.690883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1422/2817] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/users => generated 279 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:43.695725 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.710039 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fadb2a4-f7e9-4fca-a0c1-8f0c83f3f11f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.715382 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.716125 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.716413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.716678 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.717324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.718976 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.731829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1396/2818] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:43.736057 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.747664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759a5d92-6fc7-47af-a068-c3742059aa98 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.753261 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8466cf3e-5eb8-407e-92de-904bcaa6776b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.753884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466cf3e-5eb8-407e-92de-904bcaa6776b admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.754163 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466cf3e-5eb8-407e-92de-904bcaa6776b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.754470 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8466cf3e-5eb8-407e-92de-904bcaa6776b admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.755132 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466cf3e-5eb8-407e-92de-904bcaa6776b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.756736 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8466cf3e-5eb8-407e-92de-904bcaa6776b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.769309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1423/2819] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:43.774709 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:43.785235 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935c5ac-b183-4937-89a6-185b0aaac19e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:43.789831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:43.790515 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.790788 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.791056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] PATH_INFO: `/v3/projects/2b9af9ba43e24a33a80124c67c86b7d7/users/060480124205497faf530555a8ff651b/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.805954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b9af9ba43e24a33a80124c67c86b7d7, user_id=060480124205497faf530555a8ff651b, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:43.809181 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:43.823783 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1397/2820] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:43 2022] PUT /identity/v3/projects/2b9af9ba43e24a33a80124c67c86b7d7/users/060480124205497faf530555a8ff651b/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:43.835463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:43.835796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:43.836107 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:43.869249 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] MFA Rules not processed for user `060480124205497faf530555a8ff651b`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:43.870657 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:43.941150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1424/2821] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:43.945933 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.014697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8a068a-b9e2-409f-b660-026ad54d549a admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.028241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.028863 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.029141 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.029418 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.030051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.032058 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.063263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1398/2822] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:53:43 2022] POST /identity/v3/projects => generated 357 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:44.067678 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.079568 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f4b2ff-6ff0-444f-877b-aa04ef62ea8b None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.142555 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.143171 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.143536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.143837 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.144464 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.146492 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.157091 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:44.175454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1425/2823] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:53:44 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:53:44.179529 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.190322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5472135-0bda-4da6-92bd-3b95776db037 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.195216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.195816 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.196094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.196363 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.196995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.199114 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.209809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1399/2824] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:44.214805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.225571 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234987e8-270d-4361-adec-746c6e452712 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.230992 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.231865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.232238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.232599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] PATH_INFO: `/v3/projects/0b966d9fb21f4095a73757d1d6831bb2/users/6973a566c6e246d89f778d30b971b3e0/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.251936 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b966d9fb21f4095a73757d1d6831bb2, user_id=6973a566c6e246d89f778d30b971b3e0, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.254617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.270989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1426/2825] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:44 2022] PUT /identity/v3/projects/0b966d9fb21f4095a73757d1d6831bb2/users/6973a566c6e246d89f778d30b971b3e0/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:44.274964 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.285970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab58dfb-cc59-4e74-9ba0-c261bad69c06 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.343868 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e54a80-10e3-4787-a9b2-c19a6f6330a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.344476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e54a80-10e3-4787-a9b2-c19a6f6330a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.344751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e54a80-10e3-4787-a9b2-c19a6f6330a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.345044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e54a80-10e3-4787-a9b2-c19a6f6330a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.345692 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e54a80-10e3-4787-a9b2-c19a6f6330a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.347318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e54a80-10e3-4787-a9b2-c19a6f6330a1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.358408 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1400/2826] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:53:44.362839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.373176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf8e200-7d19-4ceb-a7dc-f70930b42b7f admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.377959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.378649 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.378929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.380323 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] PATH_INFO: `/v3/projects/0b966d9fb21f4095a73757d1d6831bb2/users/6973a566c6e246d89f778d30b971b3e0/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.381135 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b966d9fb21f4095a73757d1d6831bb2, user_id=6973a566c6e246d89f778d30b971b3e0, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.383814 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.395169 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1427/2827] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:53:44 2022] PUT /identity/v3/projects/0b966d9fb21f4095a73757d1d6831bb2/users/6973a566c6e246d89f778d30b971b3e0/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:44.406591 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.406877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.407168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.435914 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] MFA Rules not processed for user `6973a566c6e246d89f778d30b971b3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:44.437134 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:44.450827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:44.513196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1401/2828] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:53:44 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:44.524548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.526245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:44.583537 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e6ff5f-9ffe-4dc1-bdd6-81f3a073b00d admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.587184 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.587788 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.588056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.588326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.597819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.599280 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.609149 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1428/2829] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:44.684287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:44.694641 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d28db9-6284-4981-bd3f-d8fd351b4e7f None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:44.699049 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e496513-89dd-4233-893e-e81d595af57e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:44.699657 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e496513-89dd-4233-893e-e81d595af57e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:44.699941 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e496513-89dd-4233-893e-e81d595af57e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:44.700214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e496513-89dd-4233-893e-e81d595af57e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:44.756649 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e496513-89dd-4233-893e-e81d595af57e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:44.758877 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e496513-89dd-4233-893e-e81d595af57e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:44.769064 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1402/2830] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:44 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:53:54.604411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:INSERT=4,keystone:SELECT=43 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:54.764851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=50 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:53:59.684275 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:59.747772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc14d1-6971-460f-9901-2333641678a6 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:53:59.752337 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:53:59.752960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:59.753239 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:59.753512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] PATH_INFO: `/v3/users/060480124205497faf530555a8ff651b` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:59.754653 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=060480124205497faf530555a8ff651b)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:53:59.756559 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:53:59.806946 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '060480124205497faf530555a8ff651b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:59.807601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '060480124205497faf530555a8ff651b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:59.808208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '060480124205497faf530555a8ff651b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:59.816346 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '060480124205497faf530555a8ff651b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:59.829869 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '060480124205497faf530555a8ff651b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:53:59.843058 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1429/2831] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:53:59 2022] DELETE /identity/v3/users/060480124205497faf530555a8ff651b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:53:59.847638 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:53:59.848109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:53:59.848213 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:53:59.866852 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:53:59.868164 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:59.926171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1403/2832] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:53:59.936372 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:53:59.937776 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:53:59.996573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf676a-be49-44f3-957e-3728d8376f59 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:00.000134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:00.000728 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:00.001000 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:00.001272 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:00.010025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:00.012025 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:00.021175 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1430/2833] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:53:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:00.289845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:00.291427 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:00.306083 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7211b0-3b76-45bc-86b5-b5fbbc48e8df None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:00.312287 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:00.312929 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:00.313316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:00.313467 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] PATH_INFO: `/v3/projects/2b9af9ba43e24a33a80124c67c86b7d7` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:00.314774 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b9af9ba43e24a33a80124c67c86b7d7)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:00.316544 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:00.364527 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b9af9ba43e24a33a80124c67c86b7d7', 'request_id': 'req-b207b563-a99e-406f-8842-6aedbab73b20'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.371075 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b9af9ba43e24a33a80124c67c86b7d7', 'request_id': 'req-b207b563-a99e-406f-8842-6aedbab73b20'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.378322 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b9af9ba43e24a33a80124c67c86b7d7', 'request_id': 'req-b207b563-a99e-406f-8842-6aedbab73b20'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.389227 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] The token cache is being invalidate because project 2b9af9ba43e24a33a80124c67c86b7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:00.389508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.390130 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.391619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1404/2834] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:00 2022] DELETE /identity/v3/projects/2b9af9ba43e24a33a80124c67c86b7d7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:00.395978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:00.405563 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64721565-fd01-4443-b72f-4cac50f0356f service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:00.423609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:00.464046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:00.464598 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:00.464877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:00.465157 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] PATH_INFO: `/v3/users/6973a566c6e246d89f778d30b971b3e0` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:00.466309 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6973a566c6e246d89f778d30b971b3e0)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:00.468300 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:00.512684 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6973a566c6e246d89f778d30b971b3e0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.513332 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6973a566c6e246d89f778d30b971b3e0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.513939 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6973a566c6e246d89f778d30b971b3e0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.521326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6973a566c6e246d89f778d30b971b3e0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.533313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-d1df0b13-f213-4f9a-8bcc-cbb5e370a461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6973a566c6e246d89f778d30b971b3e0'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.546223 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1431/2835] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:00 2022] DELETE /identity/v3/users/6973a566c6e246d89f778d30b971b3e0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:00.811301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:00.812992 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:00.834165 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207b563-a99e-406f-8842-6aedbab73b20 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:00.891693 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:00.892401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:00.892590 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:00.892994 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] PATH_INFO: `/v3/projects/0b966d9fb21f4095a73757d1d6831bb2` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:00.894021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b966d9fb21f4095a73757d1d6831bb2)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:00.895973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:00.926532 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:00.944106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b966d9fb21f4095a73757d1d6831bb2', 'request_id': 'req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.949470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b966d9fb21f4095a73757d1d6831bb2', 'request_id': 'req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.957156 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b966d9fb21f4095a73757d1d6831bb2', 'request_id': 'req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.968743 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] The token cache is being invalidate because project 0b966d9fb21f4095a73757d1d6831bb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:00.969030 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.969656 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-df4bd8cb-83a1-4c48-a38b-12ef89a8a759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:00.971173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1405/2836] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:00 2022] DELETE /identity/v3/projects/0b966d9fb21f4095a73757d1d6831bb2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:00.976538 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:00.977056 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:00.977176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:00.996912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:00.998116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:01.056984 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1432/2837] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:01.064912 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.065400 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.065707 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.085974 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:01.087204 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:01.095792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1406/2838] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:01.101233 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.115718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdde00c-220a-45c2-857e-1a4565616f04 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.120367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.120968 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.121245 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.121516 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.122156 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:01.122486 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.124082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.135575 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1433/2839] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:01.142093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.151665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546f26f1-0664-4d5d-92f9-18387daa5103 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.156937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.157554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.157831 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.158101 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.158768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.160697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.187957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1407/2840] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:01.192495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.202975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a40e94-7af5-4dd9-93de-ce9ee6f6c543 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.228544 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:01.262945 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.263469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.263741 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.264035 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.264628 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.266551 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.288082 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1434/2841] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/users => generated 291 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:01.292302 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.303096 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a4fae6-291e-4c93-a32b-3e06308de7ce admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.307747 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.308348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.308626 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.308896 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.309536 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.311176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.321178 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1408/2842] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:01.325690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.335609 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa36c76-135c-465e-bfbf-2ba75d17f4ea admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.340830 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d82537-1110-445d-8222-ad2fa9581949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.341425 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d82537-1110-445d-8222-ad2fa9581949 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.341703 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d82537-1110-445d-8222-ad2fa9581949 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.341974 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d82537-1110-445d-8222-ad2fa9581949 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.342679 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d82537-1110-445d-8222-ad2fa9581949 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.344326 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d82537-1110-445d-8222-ad2fa9581949 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.355076 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1435/2843] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:01.359243 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.369330 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cffb2c7-e643-4a03-9335-6937fbace05d admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.373952 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.374621 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.374922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.375172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] PATH_INFO: `/v3/projects/a2b91d8e03804db4af63148c52c019fe/users/609fd42bf88548afb74030a139196725/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.390001 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2b91d8e03804db4af63148c52c019fe, user_id=609fd42bf88548afb74030a139196725, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.392706 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.406015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1409/2844] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:01 2022] PUT /identity/v3/projects/a2b91d8e03804db4af63148c52c019fe/users/609fd42bf88548afb74030a139196725/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:01.416182 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.416468 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.416738 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.444378 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] MFA Rules not processed for user `609fd42bf88548afb74030a139196725`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:01.445588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:01.508857 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1436/2845] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:01.514122 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.576154 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d556c7-6e4c-4cc9-a544-e6975b1f9312 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.580843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.581447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.581730 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.582006 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.582686 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.584601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.610807 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1410/2846] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/projects => generated 383 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:01.615045 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.626150 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc1b607-725e-4a50-8b7a-051477af58d8 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.681505 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.681879 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.682295 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.682591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.683716 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.685619 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.707630 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1437/2847] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/users => generated 298 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:01.711665 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.721880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67579f7-257a-4a4d-9917-8be565b7e2c4 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.726683 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.727286 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.727568 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.727843 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.728484 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.730120 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.740587 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1411/2848] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:01.744584 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.754813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a20bf6-f642-4542-b46d-4432e450495a admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.759370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.760016 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.760292 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.760561 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] PATH_INFO: `/v3/projects/af9368fe27bc47b18851cb3575b275cf/users/546e0ea393844880be02c2fab0659db3/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.775443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9368fe27bc47b18851cb3575b275cf, user_id=546e0ea393844880be02c2fab0659db3, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.778042 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.791188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1438/2849] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:01 2022] PUT /identity/v3/projects/af9368fe27bc47b18851cb3575b275cf/users/546e0ea393844880be02c2fab0659db3/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:01.795104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.804957 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676a3a01-b212-41c3-a6ef-814ae37709a1 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.825482 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:01.865732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f89909-687c-456a-a45d-949158cafd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.866368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f89909-687c-456a-a45d-949158cafd39 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.866648 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f89909-687c-456a-a45d-949158cafd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.866920 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f89909-687c-456a-a45d-949158cafd39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.867554 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f89909-687c-456a-a45d-949158cafd39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.869146 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f89909-687c-456a-a45d-949158cafd39 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.879603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1412/2850] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:01 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:01.883760 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:01.889214 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:01.895409 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda5730f-6ba6-4a34-a7a8-da8a14347edb admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:01.900535 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:01.901186 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.901493 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.901762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] PATH_INFO: `/v3/projects/af9368fe27bc47b18851cb3575b275cf/users/546e0ea393844880be02c2fab0659db3/roles/4edad00f66404dc7bc651b8ed8896064` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.903690 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af9368fe27bc47b18851cb3575b275cf, user_id=546e0ea393844880be02c2fab0659db3, role_id=4edad00f66404dc7bc651b8ed8896064)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:01.906373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:01.917019 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1439/2851] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:01 2022] PUT /identity/v3/projects/af9368fe27bc47b18851cb3575b275cf/users/546e0ea393844880be02c2fab0659db3/roles/4edad00f66404dc7bc651b8ed8896064 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:01.927871 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:01.928389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:01.928389 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:01.957421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] MFA Rules not processed for user `546e0ea393844880be02c2fab0659db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:01.958675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.028309 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1413/2852] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:01 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:02.039041 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.040787 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:02.098699 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb495312-52e6-4f36-bdbf-026c209e7434 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:02.102301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:02.102914 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.103191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.103463 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.113284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:02.114822 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:02.124907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1440/2853] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:02 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:02.225104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:02.235296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd203bd2-c0e8-4ce1-9deb-640517702e54 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:02.238845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63347ac5-8193-45bf-8370-dd1a24c05b62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:02.239434 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63347ac5-8193-45bf-8370-dd1a24c05b62 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:02.239711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63347ac5-8193-45bf-8370-dd1a24c05b62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:02.239979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63347ac5-8193-45bf-8370-dd1a24c05b62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:02.298845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63347ac5-8193-45bf-8370-dd1a24c05b62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:02.300217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63347ac5-8193-45bf-8370-dd1a24c05b62 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:02.309007 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1414/2854] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:04.880824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:04.941581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db639107-9215-45d4-a077-e7fd0c19a014 service cinder] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:04.946359 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:04.946978 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:04.947255 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:04.947526 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] PATH_INFO: `/v3/users/609fd42bf88548afb74030a139196725` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:04.948719 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] RBAC: Authorizing `identity:delete_user(user_id=609fd42bf88548afb74030a139196725)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:04.950659 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:04.995883 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '609fd42bf88548afb74030a139196725'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.996522 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '609fd42bf88548afb74030a139196725'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:04.997114 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '609fd42bf88548afb74030a139196725'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.005602 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '609fd42bf88548afb74030a139196725'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.017503 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '609fd42bf88548afb74030a139196725'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.029455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1441/2855] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:04 2022] DELETE /identity/v3/users/609fd42bf88548afb74030a139196725 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:05.034734 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.035214 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.035936 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.055133 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:05.056348 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.102954 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=100 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:05.122901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1415/2856] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:05.132456 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.133827 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.152629 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:05.198754 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ae4ac4-d731-43f2-9240-0d0989f81343 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.202411 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.203010 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.203286 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.203558 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.212884 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.214844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.224259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1442/2857] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:05.488552 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.490084 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:05.502358 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb2cfe4-9e07-465c-bae2-3bc8c6bcb8b7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.507074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.507824 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.508174 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.508516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] PATH_INFO: `/v3/projects/a2b91d8e03804db4af63148c52c019fe` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.509722 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2b91d8e03804db4af63148c52c019fe)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.511771 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.559903 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2b91d8e03804db4af63148c52c019fe', 'request_id': 'req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.565359 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2b91d8e03804db4af63148c52c019fe', 'request_id': 'req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.572951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2b91d8e03804db4af63148c52c019fe', 'request_id': 'req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.584739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] The token cache is being invalidate because project a2b91d8e03804db4af63148c52c019fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:05.585028 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.585688 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.587196 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1416/2858] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:05 2022] DELETE /identity/v3/projects/a2b91d8e03804db4af63148c52c019fe => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:05.591740 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:05.601499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8f46-d707-46f5-aa82-f20a5b6ff96c service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:05.657819 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:05.658499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:05.658772 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:05.659043 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] PATH_INFO: `/v3/users/546e0ea393844880be02c2fab0659db3` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:05.660146 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=546e0ea393844880be02c2fab0659db3)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:05.662020 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:05.707665 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '546e0ea393844880be02c2fab0659db3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.708319 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '546e0ea393844880be02c2fab0659db3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.708927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '546e0ea393844880be02c2fab0659db3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.716662 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '546e0ea393844880be02c2fab0659db3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.728420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ea2cedf4-f1e6-4cfe-8682-95431ce3fa48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '546e0ea393844880be02c2fab0659db3'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:05.740513 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1443/2859] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:05 2022] DELETE /identity/v3/users/546e0ea393844880be02c2fab0659db3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:06.013950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.015502 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:06.028979 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12de465-b6dc-47a2-ac69-6a31bfc5ec27 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.088596 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.089282 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.089494 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.089768 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] PATH_INFO: `/v3/projects/af9368fe27bc47b18851cb3575b275cf` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.090970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af9368fe27bc47b18851cb3575b275cf)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.092899 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.139837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af9368fe27bc47b18851cb3575b275cf', 'request_id': 'req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:06.145413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af9368fe27bc47b18851cb3575b275cf', 'request_id': 'req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:06.153085 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af9368fe27bc47b18851cb3575b275cf', 'request_id': 'req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:06.160173 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:06.165818 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] The token cache is being invalidate because project af9368fe27bc47b18851cb3575b275cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:06.166325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:06.167450 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-26b50d9d-f5b4-450e-ab9c-6b0e1f1993a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:06.169092 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1417/2860] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:06 2022] DELETE /identity/v3/projects/af9368fe27bc47b18851cb3575b275cf => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:06.174831 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.175340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.175454 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.195340 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:06.196533 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:06.225439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:06.257024 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1444/2861] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:06.319332 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.319616 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.319894 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.340036 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:06.341252 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:06.349264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1418/2862] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:06.353473 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.363585 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734da824-afb7-4a12-9e9a-31a1db5ab6bf None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.368127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.368726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.369007 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.369277 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] PATH_INFO: `/v3/domains` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.369920 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81521) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:06.370796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.372384 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.383725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1445/2863] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:06.387797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.396882 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0da946-9414-4aeb-979e-d8c55fd7b0b4 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.401540 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.402153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.402458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.402726 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.403383 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.405263 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.433758 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1419/2864] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:06.437927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.448219 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169f5c5-4927-44bc-bafc-16ff78783b76 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.504113 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.504709 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.504988 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.505263 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] PATH_INFO: `/v3/users` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.505886 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.507823 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.528536 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1446/2865] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:06.532644 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.544027 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81a6a71-22d7-49ce-9f61-b577b8cd6d99 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.548794 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.549397 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.549676 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.549951 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.550627 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.552234 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.562827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1420/2866] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:06.567160 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.577369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f9a153-7bff-40fc-a245-7705c593af70 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.581954 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90afa4c5-2426-4878-a33c-64fc8ba32419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.582577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afa4c5-2426-4878-a33c-64fc8ba32419 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.582852 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afa4c5-2426-4878-a33c-64fc8ba32419 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.583118 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90afa4c5-2426-4878-a33c-64fc8ba32419 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.583749 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afa4c5-2426-4878-a33c-64fc8ba32419 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.585321 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90afa4c5-2426-4878-a33c-64fc8ba32419 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.595706 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1447/2867] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:06.599802 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.610436 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910e8551-fd3e-484f-b4d6-fabea864f673 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.615094 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.615740 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.616020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.616294 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] PATH_INFO: `/v3/projects/1ccad89b0b2a4247894e6b4cb2d60354/users/466874a8f6d0445896999e800266f10c/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.631109 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ccad89b0b2a4247894e6b4cb2d60354, user_id=466874a8f6d0445896999e800266f10c, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.633777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.646256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1421/2868] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:06 2022] PUT /identity/v3/projects/1ccad89b0b2a4247894e6b4cb2d60354/users/466874a8f6d0445896999e800266f10c/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:06.656046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.656259 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.656528 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.685154 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] MFA Rules not processed for user `466874a8f6d0445896999e800266f10c`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:06.686387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:06.750614 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1448/2869] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:06.762510 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:06.764200 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:06.841853 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4c087-6766-4f21-bc95-ae04ff2b6a9e admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:06.846163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:06.846869 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:06.847158 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:06.847508 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:06.856738 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:06.858205 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:06.866858 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1422/2870] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:06 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:16.747601 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:16.862971 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:30.780142 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:30.781766 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:30.924474 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597cde3-d8f7-4e4f-aec4-2145c4e7f41a None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:30.929421 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:30.930103 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:30.930405 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:30.930672 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:30.932387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:30.944591 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:30.946143 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7f95-f033-413d-8dad-dc0ee4545235 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:30.956345 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1449/2871] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:40.952179 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=63 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:52.803074 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:52.881250 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089a940d-1eaa-420f-8146-fa21be9f5f7a service glance] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:52.885837 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:52.886486 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:52.886775 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:52.887047 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] PATH_INFO: `/v3/users/466874a8f6d0445896999e800266f10c` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:52.888177 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=466874a8f6d0445896999e800266f10c)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:52.890104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:52.937797 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466874a8f6d0445896999e800266f10c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.938516 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466874a8f6d0445896999e800266f10c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.939140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466874a8f6d0445896999e800266f10c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.946242 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466874a8f6d0445896999e800266f10c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.953009 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466874a8f6d0445896999e800266f10c'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:52.971015 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1423/2872] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:52 2022] DELETE /identity/v3/users/466874a8f6d0445896999e800266f10c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:52.975581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:52.976098 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:52.976221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:52.994658 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:52.995893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:53.055081 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1450/2873] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:53.065316 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.067040 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:53.124390 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183987de-6bf1-4654-94e6-b37aa983c0fd admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.128566 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b231ecf-82fe-4058-9d42-4d981eb7345b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.129171 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b231ecf-82fe-4058-9d42-4d981eb7345b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.129452 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b231ecf-82fe-4058-9d42-4d981eb7345b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.129731 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b231ecf-82fe-4058-9d42-4d981eb7345b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.138821 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b231ecf-82fe-4058-9d42-4d981eb7345b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.140267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b231ecf-82fe-4058-9d42-4d981eb7345b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.150724 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1424/2874] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:53.422231 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.428520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:53.449726 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c8e3b4-0637-485c-be99-48de8dda7605 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.454820 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.455373 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.455650 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.455942 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] PATH_INFO: `/v3/projects/1ccad89b0b2a4247894e6b4cb2d60354` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.457074 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ccad89b0b2a4247894e6b4cb2d60354)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.458996 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.507062 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ccad89b0b2a4247894e6b4cb2d60354', 'request_id': 'req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.512502 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ccad89b0b2a4247894e6b4cb2d60354', 'request_id': 'req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.518620 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ccad89b0b2a4247894e6b4cb2d60354', 'request_id': 'req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.530608 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] The token cache is being invalidate because project 1ccad89b0b2a4247894e6b4cb2d60354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:53.530889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.531497 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-0c841a3e-da88-4b56-9d5a-f0ba2d598b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:53.533033 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1451/2875] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:53 2022] DELETE /identity/v3/projects/1ccad89b0b2a4247894e6b4cb2d60354 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:53.538241 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.538528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.538792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.547264 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:53.559310 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:53.561041 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:53.624685 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1425/2876] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:53.633593 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.633875 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.634147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.653664 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:53.654890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:53.663265 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1452/2877] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:53.667595 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.677803 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b9b4aa-f2f6-4f1b-b9d2-21b327646ec0 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.682336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.682922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.683195 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.683462 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.684105 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:53.684420 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.685998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.697556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1426/2878] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:53.701905 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.712017 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc763e17-15ab-4d58-bc06-58955b9b8d34 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.718839 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.720210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.720488 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.720762 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.721400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.723342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.751338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1453/2879] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:53 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:53.755671 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.765699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83e82d-1a13-46d6-a130-5da0e6a9414f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.822160 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.822854 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.822982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.823224 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.823847 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.825756 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.847567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1427/2880] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:53 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:53.852210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.862410 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cad1961-a07c-4931-804c-362f149a1423 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.866989 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.867582 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.867858 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.868131 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.868765 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.870380 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.880870 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1454/2881] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 24 03:54:53.884790 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.895301 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9c9b7-b329-4fa1-9fbc-ea5471a94f28 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.899973 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aefa563-d56b-4b48-8e1b-1fe4275ff4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.900564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa563-d56b-4b48-8e1b-1fe4275ff4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.900842 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa563-d56b-4b48-8e1b-1fe4275ff4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.901112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aefa563-d56b-4b48-8e1b-1fe4275ff4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.901751 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefa563-d56b-4b48-8e1b-1fe4275ff4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.903368 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aefa563-d56b-4b48-8e1b-1fe4275ff4b4 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.913639 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1428/2882] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:53.917675 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:53.927661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb792764-4f97-456a-9ceb-b58ff0c2b091 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:53.932210 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:53.932844 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.933116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.933577 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] PATH_INFO: `/v3/projects/d5bf584b9033476f9de1a676dd2cc072/users/afce10a99a1a4f74a439f02909289fe3/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:53.949176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d5bf584b9033476f9de1a676dd2cc072, user_id=afce10a99a1a4f74a439f02909289fe3, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:53.951872 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:53.965208 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1455/2883] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:53 2022] PUT /identity/v3/projects/d5bf584b9033476f9de1a676dd2cc072/users/afce10a99a1a4f74a439f02909289fe3/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 24 03:54:53.975745 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:53.976051 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:53.976345 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:54.005220 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] MFA Rules not processed for user `afce10a99a1a4f74a439f02909289fe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:54.006470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:54.067325 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:54.071592 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1429/2884] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:54.083975 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:54.085805 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:54.112134 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:54.149124 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973b0241-caec-470f-80fc-4b475e464319 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:54.152778 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cec69bd-cdef-4d5e-bc3e-28a14d5480e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:54.153379 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec69bd-cdef-4d5e-bc3e-28a14d5480e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:54.153660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec69bd-cdef-4d5e-bc3e-28a14d5480e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:54.153937 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cec69bd-cdef-4d5e-bc3e-28a14d5480e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:54.162376 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec69bd-cdef-4d5e-bc3e-28a14d5480e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:54.163813 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cec69bd-cdef-4d5e-bc3e-28a14d5480e3 service cinder] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:54.173313 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1456/2885] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:56.722446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.802126 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbe52cb-7cb2-4f30-8127-63d0193849c2 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:56.806634 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:56.807256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.807535 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.807805 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] PATH_INFO: `/v3/users/afce10a99a1a4f74a439f02909289fe3` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.808919 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] RBAC: Authorizing `identity:delete_user(user_id=afce10a99a1a4f74a439f02909289fe3)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:56.810958 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:56.858258 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afce10a99a1a4f74a439f02909289fe3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:56.858918 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afce10a99a1a4f74a439f02909289fe3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:56.859525 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afce10a99a1a4f74a439f02909289fe3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:56.867054 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afce10a99a1a4f74a439f02909289fe3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:56.874277 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afce10a99a1a4f74a439f02909289fe3'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:56.891217 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1430/2886] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:54:56 2022] DELETE /identity/v3/users/afce10a99a1a4f74a439f02909289fe3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:56.895796 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:56.896284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:56.896400 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:56.927825 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:56.929044 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:56.988579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1457/2887] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 24 03:54:56.998318 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:56.999765 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.065603 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21eabc23-5b71-4d89-887a-179678825712 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.069564 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35b27ed-f25c-44b5-94c1-d4d1f83d435b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.070157 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b27ed-f25c-44b5-94c1-d4d1f83d435b service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.070465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b27ed-f25c-44b5-94c1-d4d1f83d435b service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.070732 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35b27ed-f25c-44b5-94c1-d4d1f83d435b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.077497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:57.080763 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b27ed-f25c-44b5-94c1-d4d1f83d435b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.082935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35b27ed-f25c-44b5-94c1-d4d1f83d435b service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.094061 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1431/2888] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:54:57.357976 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.359427 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.371725 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6583acd7-035a-4b4e-8f49-255c00294248 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.377004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.377643 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.377927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.378220 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] PATH_INFO: `/v3/projects/d5bf584b9033476f9de1a676dd2cc072` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.379342 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d5bf584b9033476f9de1a676dd2cc072)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.381240 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.428198 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd5bf584b9033476f9de1a676dd2cc072', 'request_id': 'req-af740b26-d363-42db-a0a3-9be0deb0ef8b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:57.433581 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd5bf584b9033476f9de1a676dd2cc072', 'request_id': 'req-af740b26-d363-42db-a0a3-9be0deb0ef8b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:57.439745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd5bf584b9033476f9de1a676dd2cc072', 'request_id': 'req-af740b26-d363-42db-a0a3-9be0deb0ef8b'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:57.452573 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] The token cache is being invalidate because project d5bf584b9033476f9de1a676dd2cc072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:54:57.452862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:57.453509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-af740b26-d363-42db-a0a3-9be0deb0ef8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:54:57.455590 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1458/2889] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:54:57 2022] DELETE /identity/v3/projects/d5bf584b9033476f9de1a676dd2cc072 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:57.460414 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.460699 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.460982 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.481192 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:57.482439 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.541093 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1432/2890] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:57.548780 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.549065 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.549338 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.569540 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:57.570773 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.579036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1459/2891] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:57.583793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.593565 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd9fe2d-bc7d-4866-83a9-9b29c5860fa7 None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.598162 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.598792 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.599072 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.599344 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.599995 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81522) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 24 03:54:57.600296 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.601966 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.613354 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1433/2892] 10.222.0.91 () {58 vars in 1153 bytes} [Wed Aug 24 03:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 24 03:54:57.617616 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.627258 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16606bb8-ecd5-406f-99bc-69f9cda449ab None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.631894 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.632512 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.632790 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.633061 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.633686 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.635595 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.664673 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1460/2893] 10.222.0.91 () {60 vars in 1153 bytes} [Wed Aug 24 03:54:57 2022] POST /identity/v3/projects => generated 363 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 24 03:54:57.668916 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.679650 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42099e20-84eb-455c-91a9-3010670eacdb admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.735175 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.735880 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.736625 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.736898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] PATH_INFO: `/v3/users` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.737528 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.739559 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.760048 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1434/2894] 10.222.0.91 () {60 vars in 1144 bytes} [Wed Aug 24 03:54:57 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 24 03:54:57.764462 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.774718 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d7687-ef09-4346-b101-2b62d2776ff3 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.779315 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.779912 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.780188 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.780458 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] PATH_INFO: `/v3/roles` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.781092 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.783238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.784457 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:57.795284 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1461/2895] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:57.799417 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.809523 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034a1fc-8fc8-4525-bfba-f969e3864d4f admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.814117 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5a3c6-7529-416b-b193-77213224506f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.814742 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5a3c6-7529-416b-b193-77213224506f admin admin] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.815020 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5a3c6-7529-416b-b193-77213224506f admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.815312 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5a3c6-7529-416b-b193-77213224506f admin admin] PATH_INFO: `/v3/roles` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.816208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5a3c6-7529-416b-b193-77213224506f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.817898 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5a3c6-7529-416b-b193-77213224506f admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.828262 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1435/2896] 10.222.0.91 () {58 vars in 1122 bytes} [Wed Aug 24 03:54:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 24 03:54:57.833046 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:57.843003 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea183bb-78ad-483f-9eb3-87ed60b53992 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:57.848287 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:57.848929 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.849226 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.849500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] PATH_INFO: `/v3/projects/b5f4bbb0be13428a8ad3d432acc8a7e5/users/af47b4e5851842d9bf63a2f2e417e96e/roles/d89a893a140749d39b9cecdedae58d78` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.864191 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5f4bbb0be13428a8ad3d432acc8a7e5, user_id=af47b4e5851842d9bf63a2f2e417e96e, role_id=d89a893a140749d39b9cecdedae58d78)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:57.866888 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:57.879542 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1462/2897] 10.222.0.91 () {60 vars in 1483 bytes} [Wed Aug 24 03:54:57 2022] PUT /identity/v3/projects/b5f4bbb0be13428a8ad3d432acc8a7e5/users/af47b4e5851842d9bf63a2f2e417e96e/roles/d89a893a140749d39b9cecdedae58d78 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:54:57.890514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1004-a60d-4bce-8468-1f9da8ca752f None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:57.890823 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1004-a60d-4bce-8468-1f9da8ca752f None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:57.891098 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f1004-a60d-4bce-8468-1f9da8ca752f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:57.920256 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-741f1004-a60d-4bce-8468-1f9da8ca752f None None] MFA Rules not processed for user `af47b4e5851842d9bf63a2f2e417e96e`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:54:57.921476 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-741f1004-a60d-4bce-8468-1f9da8ca752f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:57.969739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:54:57.987759 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1436/2898] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:54:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:54:57.999187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:54:58.000890 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:54:58.061238 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9c1e7b-1689-4dc1-bc7a-cbaf965a1af4 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:54:58.065500 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d0eef76ae53400c83c391351dc75c32', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:54:58.066382 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:54:58.066661 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:54:58.066933 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:54:58.075491 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:54:58.076972 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:54:58.086241 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1463/2899] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:54:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:07.983984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=4 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:08.082110 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=38,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:09.226492 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:09.227056 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:09.227168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:09.246848 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] MFA Rules not processed for user `9c1f5f4c1e544708a69b5631af831dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:09.248059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:09.301772 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1437/2900] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:55:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:11.047120 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:11.048663 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:11.128183 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc12b5-bbde-44f2-9751-9dae08c1e2cc service glance] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:11.133369 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:11.133983 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:11.134341 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:11.134618 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:11.136351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:11.148877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:11.150530 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:11.161068 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1464/2901] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:19.294349 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=25 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:21.156201 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=32 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:33.924280 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:33.926465 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:34.012711 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9b27d-b6cf-4d66-b600-1adc7edbc38d None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:34.018336 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:34.019619 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:34.019993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:34.020267 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:34.022044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:34.035339 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:34.036935 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a6fba1-fd07-4a59-9b61-228c3ba7a8f1 service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:34.047984 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1438/2902] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:35.200079 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:35.262367 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b332f6b-227c-45bb-a261-8c8ec21b2764 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:35.266932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:35.267548 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:35.267838 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:35.268112 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] PATH_INFO: `/v3/users/af47b4e5851842d9bf63a2f2e417e96e` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:35.269222 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af47b4e5851842d9bf63a2f2e417e96e)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:35.271694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:35.318351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af47b4e5851842d9bf63a2f2e417e96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.319004 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af47b4e5851842d9bf63a2f2e417e96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.319613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af47b4e5851842d9bf63a2f2e417e96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.327006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af47b4e5851842d9bf63a2f2e417e96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.338720 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af47b4e5851842d9bf63a2f2e417e96e'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.350370 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1465/2903] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 03:55:35 2022] DELETE /identity/v3/users/af47b4e5851842d9bf63a2f2e417e96e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:35.354833 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] REQUEST_METHOD: `POST` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:35.355327 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:35.355447 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:35.383514 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.auth.core [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] MFA Rules not processed for user `7d825b9d979b45a095dd6e2cabfe9971`. Rule list: `[]` (Enabled: `True`). {{(pid=81522) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:55:35.384652 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:35.441953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1439/2904] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:55:35.451734 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:35.453116 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:35.524095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2cb278-f1db-42d8-afdb-26accb70bbc7 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:35.527979 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:35.528589 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:35.528873 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:35.529147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:35.538927 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:35.540349 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:35.549660 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1466/2905] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:35.887991 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:35.889633 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:35.901922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae800014-4093-4bec-a251-99e303cf6d2a None None] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:35.907324 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:35.907953 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:35.908229 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:35.908493 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] PATH_INFO: `/v3/projects/b5f4bbb0be13428a8ad3d432acc8a7e5` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:35.909609 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5f4bbb0be13428a8ad3d432acc8a7e5)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:35.912077 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:35.959059 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5f4bbb0be13428a8ad3d432acc8a7e5', 'request_id': 'req-b44f4769-a8ff-40f5-bfad-eea6378c85b7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.964579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5f4bbb0be13428a8ad3d432acc8a7e5', 'request_id': 'req-b44f4769-a8ff-40f5-bfad-eea6378c85b7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.972031 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5f4bbb0be13428a8ad3d432acc8a7e5', 'request_id': 'req-b44f4769-a8ff-40f5-bfad-eea6378c85b7'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.974401 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:35.985455 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] The token cache is being invalidate because project b5f4bbb0be13428a8ad3d432acc8a7e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81522) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 03:55:35.985739 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.986385 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 03:55:35.988088 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1440/2906] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 03:55:35 2022] DELETE /identity/v3/projects/b5f4bbb0be13428a8ad3d432acc8a7e5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 03:55:38.941279 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:55:38.943290 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:38.957095 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:39.021745 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa96b8-f9b7-4992-87e8-aece93dede8d service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:55:39.087252 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:55:39.087865 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:55:39.088148 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:55:39.088420 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:55:39.094147 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:55:39.118055 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:55:39.119694 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e805d-8ed0-443e-8362-2b917fdef266 service nova] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:55:39.140689 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1467/2907] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:55:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:55:45.984273 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:55:49.137824 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=66 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:27.963551 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:27.964993 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:28.021247 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44f4769-a8ff-40f5-bfad-eea6378c85b7 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:28.025392 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:28.025986 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:28.026292 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:28.026563 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:28.028444 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:28.089922 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:28.091421 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:28.100945 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1441/2908] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:38.096567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=48 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:39.113077 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] REQUEST_METHOD: `POST` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:39.113534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:39.113644 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:39.132520 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.auth.core [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] MFA Rules not processed for user `9c1f5f4c1e544708a69b5631af831dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81521) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 24 03:57:39.133724 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:39.141436 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1468/2909] 10.222.0.91 () {58 vars in 958 bytes} [Wed Aug 24 03:57:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 24 03:57:39.181388 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:39.191426 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab880a-0efb-4789-bd12-6ce8bbf08234 service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:39.195208 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '408b15d1a55a439c93a2f879c0d1cc31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:39.195829 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:39.196351 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:39.196505 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:39.198927 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:39.213070 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:39.214567 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:39.225950 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1442/2910] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:39.527967 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:39.529391 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:39.587960 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735fbd6b-56b7-4874-9a48-d264355ef798 None None] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:39.591617 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:39.592195 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:39.592469 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:39.592737 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:39.594495 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:39.604932 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:39.606383 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:39.614566 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1469/2911] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:39 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:40.083949 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:40.085399 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:40.098943 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7863095-9193-4497-b9fe-aa065bade57e service cinder] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:40.103069 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:40.103937 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:40.104963 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:40.105317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:40.107411 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:40.122556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:40.124106 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:40.136172 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1443/2912] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:41.870387 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:41.880181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e05790f-6939-4692-891e-9afd25575d01 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:41.884913 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:41.886344 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:41.886727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:41.887102 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:41.889089 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:41.900499 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:41.902613 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:41.912155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1470/2913] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:42.560762 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:42.562313 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:42.577832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa60e3-4a5e-40a0-ba09-ab230db88e74 service neutron] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:42.584808 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d56ed96c2f546e59e473b1d62bf99eb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:42.585446 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:42.585697 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:42.585968 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:42.587828 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:42.601443 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:42.603097 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:42.615670 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1444/2914] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:57:51.908683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=35 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:52.610601 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=18 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:57:57.935306 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:57:57.944237 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a8f50-ee8f-4d98-9024-9055798f035a service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:57:57.947859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:57:57.948455 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:57:57.948733 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:57:57.949006 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:57:57.950727 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:57:58.012683 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:57:58.014101 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:57:58.023547 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1471/2915] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:57:57 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:58:08.019415 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=30 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 03:59:38.764752 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 03:59:38.766140 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:38.834010 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9bfe8d-82a5-4570-9369-00a1f07982ad service nova] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 03:59:38.838981 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 03:59:38.839574 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 03:59:38.839849 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 03:59:38.840118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 03:59:38.841799 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 03:59:38.853793 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 03:59:38.855485 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 03:59:38.866168 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1445/2916] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 03:59:38 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 03:59:48.861409 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=32 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:15.438262 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:15.447862 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130c946a-eca7-404e-91a2-d0d632fed3f8 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:15.451439 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5ae7930c17434ae1b6df030b0599e768', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:15.452039 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:15.452318 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:15.452588 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:15.454404 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:15.522418 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:15.523859 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:15.533168 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1472/2917] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:18.635809 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:18.639614 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:18.657520 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9588ce-f5ff-405f-b79e-25038114d410 service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:18.667581 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:18.669000 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:18.669517 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:18.670021 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:18.673163 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:18.696413 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:18.698578 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:18.726489 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1446/2918] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:21.474407 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:21.487350 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680fa113-f854-4269-98e2-a66b47bb69e4 service neutron] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:21.493475 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:21.494443 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] REQUEST_METHOD: `GET` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:21.494900 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:21.495187 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:21.497221 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:21.510959 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:21.512893 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:21.524549 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1473/2919] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 04:00:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:00:21.548659 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:21.553970 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:21.576199 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aacbd76-e229-48a5-9c75-e7b0f5710c4d service placement] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:21.580827 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:21.581454 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:21.581728 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:21.581998 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] PATH_INFO: `/v3/users/9c1f5f4c1e544708a69b5631af831dd0` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:21.583153 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c1f5f4c1e544708a69b5631af831dd0)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:21.585104 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:21.632430 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c1f5f4c1e544708a69b5631af831dd0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.633081 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c1f5f4c1e544708a69b5631af831dd0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.633708 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c1f5f4c1e544708a69b5631af831dd0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.640675 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c1f5f4c1e544708a69b5631af831dd0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.647845 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c1f5f4c1e544708a69b5631af831dd0'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.665470 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1447/2920] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:21 2022] DELETE /identity/v3/users/9c1f5f4c1e544708a69b5631af831dd0 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:21.838940 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:21.852910 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95b08c-e545-4a02-992e-8ad1ad9ace17 service placement] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:21.919919 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:21.920579 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:21.920855 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:21.921127 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] PATH_INFO: `/v3/projects/7b550db4de5841e8a1ffcc7213275499` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:21.922301 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b550db4de5841e8a1ffcc7213275499)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:21.924181 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:21.975357 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b550db4de5841e8a1ffcc7213275499', 'request_id': 'req-b67a19db-10a5-4765-8e83-87f3ed0169a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.980798 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b550db4de5841e8a1ffcc7213275499', 'request_id': 'req-b67a19db-10a5-4765-8e83-87f3ed0169a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.986907 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b550db4de5841e8a1ffcc7213275499', 'request_id': 'req-b67a19db-10a5-4765-8e83-87f3ed0169a1'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:21.999791 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] The token cache is being invalidate because project 7b550db4de5841e8a1ffcc7213275499 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:22.000070 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.000692 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.002155 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1474/2921] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:21 2022] DELETE /identity/v3/projects/7b550db4de5841e8a1ffcc7213275499 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:22.006279 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:22.008216 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:22.021191 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b189019c-00da-4d67-84ff-1a6d890d4436 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:22.079112 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:22.079741 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.080044 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.080317 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] PATH_INFO: `/v3/users/d9275b375e664ebfa33098a165751745` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.081453 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9275b375e664ebfa33098a165751745)` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:22.083415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:22.132901 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9275b375e664ebfa33098a165751745'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.133579 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9275b375e664ebfa33098a165751745'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.134209 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9275b375e664ebfa33098a165751745'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.141118 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9275b375e664ebfa33098a165751745'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.144636 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:22.148548 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.notifications [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9275b375e664ebfa33098a165751745'} {{(pid=81522) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.167176 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1448/2922] 10.222.0.91 () {58 vars in 1224 bytes} [Wed Aug 24 04:00:22 2022] DELETE /identity/v3/users/d9275b375e664ebfa33098a165751745 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:22.353036 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Authenticating user token {{(pid=81521) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:00:22.354534 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.fernet_utils [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81521) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:00:22.369330 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a19db-10a5-4765-8e83-87f3ed0169a1 admin admin] Validating token access rules against request {{(pid=81521) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:00:22.377877 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:22.436232 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7d825b9d979b45a095dd6e2cabfe9971', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '24a8303d4cc94c8ebcfdb44c0be112a5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81521) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:00:22.436863 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:00:22.437176 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:00:22.437446 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] PATH_INFO: `/v3/projects/4abac066f9d94181bbe793fc1e6f38a4` {{(pid=81521) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:00:22.438599 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4abac066f9d94181bbe793fc1e6f38a4)` {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:00:22.440509 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] RBAC: Authorization granted {{(pid=81521) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:00:22.488115 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4abac066f9d94181bbe793fc1e6f38a4', 'request_id': 'req-ac6efbaf-e27b-4bce-813e-167c45104238'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.496351 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4abac066f9d94181bbe793fc1e6f38a4', 'request_id': 'req-ac6efbaf-e27b-4bce-813e-167c45104238'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.502554 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4abac066f9d94181bbe793fc1e6f38a4', 'request_id': 'req-ac6efbaf-e27b-4bce-813e-167c45104238'} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.520027 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] The token cache is being invalidate because project 4abac066f9d94181bbe793fc1e6f38a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81521) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 24 04:00:22.520267 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.520889 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG keystone.notifications [None req-ac6efbaf-e27b-4bce-813e-167c45104238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81521) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 24 04:00:22.522414 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: [pid: 81521|app: 0|req: 1475/2923] 10.222.0.91 () {58 vars in 1233 bytes} [Wed Aug 24 04:00:22 2022] DELETE /identity/v3/projects/4abac066f9d94181bbe793fc1e6f38a4 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 24 04:00:32.163705 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=8,keystone:DELETE=2 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:00:32.514242 dv-cinder-854262-1-iscsi devstack@keystone.service[81521]: DEBUG dbcounter [-] [81521] Writing DB stats keystone:SELECT=46,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81521) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 24 04:04:41.735415 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Authenticating user token {{(pid=81522) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 24 04:04:41.736851 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:04:41.865124 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac6828b-7e0f-41ae-8091-ae5010526e85 admin admin] Validating token access rules against request {{(pid=81522) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 24 04:04:41.928892 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '43c98ae03bf9469bb70ae0ee2e9aecb6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '02199521c8284ce2a5b47266bc213ef5', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81522) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 24 04:04:41.929497 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] REQUEST_METHOD: `GET` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 24 04:04:41.929777 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] SCRIPT_NAME: `/identity` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 24 04:04:41.930046 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81522) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 24 04:04:41.931832 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.fernet_utils [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81522) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 24 04:04:41.943458 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 24 04:04:41.945315 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f5935-58b4-4519-81bc-5dee4ba3247f service placement] RBAC: Authorization granted {{(pid=81522) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 24 04:04:41.956556 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: [pid: 81522|app: 0|req: 1449/2924] 10.222.0.91 () {60 vars in 1365 bytes} [Wed Aug 24 04:04:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 24 04:04:51.950962 dv-cinder-854262-1-iscsi devstack@keystone.service[81522]: DEBUG dbcounter [-] [81522] Writing DB stats keystone:SELECT=81 {{(pid=81522) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}